Saturday, October 12, 2024
Apps

ChatGPT won’t let you give it instruction amnesia anymore



OpenAI is making a change to stop people from messing with custom versions of ChatGPT by making the AI forget what it’s supposed to do. Basically, when a third party uses one of OpenAI’s models, they give it instructions that teach it to operate as, for example, a customer service agent for a store or a researcher for an academic publication. However, a user could mess with the chatbot by telling it to “forget all instructions,” and that phrase would induce a kind of digital amnesia and reset the chatbot to a generic blank.

To prevent this, OpenAI researchers created a new technique called “instruction hierarchy,” which is a way to prioritize the developer’s original prompts and instructions over any potentially manipulative user-created prompts. The system instructions have the highest privilege and can’t be erased so easily anymore. If a user enters a prompt that attempts to misalign the AI’s behavior, it will be rejected, and the AI responds by stating that it cannot assist with the query.



READ SOURCE

This website uses cookies. By continuing to use this site, you accept our use of cookies.