noun
Jailbreaking an AI model means removing its safety filters and limitations, allowing it to generate unrestricted content, including harmful or illegal material.
This is typically done by manipulating the input prompts or exploiting vulnerabilities in the AI’s training data. For example, some users have tricked large language models into bypassing moderation by using coded language or indirect phrasing.
Jailbreaking changes the way AI models are used by removing ethical and legal safeguards. It can expose users to harmful content, violate platform policies, and lead to legal consequences. For instance, some AI image generators have been jailbroken to produce violent or explicit content, forcing companies to tighten security measures.
A user jailbreaks an AI chatbot by using a carefully crafted prompt to bypass its content filters, making it generate offensive responses.
In other languages
See also: prompt engineering