OpenAI has introduced GPT-Red, an internal automated red-teaming model designed to identify and remediate prompt injection vulnerabilities in GPT-5.6.
This approach aims to tackle a growing safety challenge. While human red-team exercises are valuable, they cannot generate adversarial test cases at the scale required to keep up with increasingly advanced AI systems.
Prompt injection occurs when malicious instructions are hidden within third-party content that an AI may process, such as webpages, emails, tool outputs, code repositories, and local files.
Recently, CrowdStrike unveiled 5 New Prompt Injection Techniques Challenging AI Agents, where attackers plant hidden instructions that remain dormant until a specific condition or keyword activates them.
Attackers could use this method to bypass the AI’s intended tasks, coercing it to expose sensitive information, upload files, forward credentials, or take unauthorized actions.
GPT-Red automates the testing process by sending adversarial prompts, observing the target model’s responses, and iteratively developing stronger attacks.
OpenAI has trained GPT-Red using self-play reinforcement learning, where the attacker model competes against multiple defender models in simulated threat scenarios.
GPT-Red Targets GPT-5.6
GPT-Red earns rewards for successfully triggering valid failures. In contrast, defender models are rewarded for resisting attacks while still accomplishing the original user tasks.
The training environments mimic realistic attack surfaces. Depending on the scenario, GPT-Red may manipulate content in a webpage banner, an email body, a local file, or a tool response. This setup allows the model to evaluate both direct and indirect prompt-injection risks in agentic workflows.
OpenAI reported that GPT-Red successfully compromised earlier internal and production models, including those as advanced as GPT-5.5.
The findings from GPT-Red’s attacks were then used to train GPT-5.6. The company noted that GPT-5.6 Sol exhibited six times fewer failures on its most challenging direct prompt-injection benchmark than its strongest production model from just four months earlier.
Additionally, OpenAI assessed GPT-Red on an internal replica of an indirect prompt-injection arena, based on research by Dziemian et al. GPT-Red reportedly succeeded in 84% of test scenarios against GPT-5.1, while human red teamers operating independently achieved only 13%.
In another test, GPT-Red targeted an AI-enabled vending machine agent, successfully executing malicious actions such as changing the price of high-value inventory to $0.50, adding a higher-priced item at that price, and canceling another customer’s order.
OpenAI stated that these issues were disclosed, and additional safeguards are currently being tested.
To mitigate risks, OpenAI keeps GPT-Red separate from publicly deployed models to prevent exposure of the offensive capabilities developed during its training.
The company reported that GPT-5.6 Sol now fails on only 0.05% of GPT-Red’s direct prompt-injection attempts across its controlled environments, while maintaining its general capabilities and avoiding excessive refusal behavior.
Strengthen Your SOC by Accelerating Threat Detection & Rapid Investigations. -> Integrate ANY.RUN With Your SOC Now.
