OpenAI has disclosed details of GPT-Red, an internal automated red-teaming model that scales prompt injection vulnerability discovery with an aim to fix issues before the tools are deployed widely.
“GPT‑Red is a strong red-teamer, and our previous models are highly vulnerable to its prompt injection attacks,” the artificial intelligence (AI) company said. “We use GPT‑Red to adversarially train GPT‑5.6, making it much more robust to prompt injections.”
The model works just like a human red-teamer. It sends a prompt, monitors how a GPT model responds, and iterates its way towards a malicious goal, such as uploading sensitive data to an external server.
The development comes as adversarial prompt injections continue to be a persistent thorn in the flesh of large language models, which can be tricked into executing a carefully crafted instruction that can produce undesirable consequences.
As agentic systems continue to be hooked to third-party data sources through web browsers, connected apps, local files, and other tools, they have also broadened the attack surface and presented more pathways for bad actors to influence the outcome of a model by embedding malicious prompts within seemingly harmless content that’s fed as input. This can take the form of an email, a web page, a tool response, or a code repository.
GPT-Red aims to augment human red-teaming at scale, thereby making it possible to identify new failure modes, improve robustness, and build suitable countermeasures before the models can be deployed.
“Similar to how human red-teamers craft attacks, the model works toward a goal by sending a prompt, observing how GPT models respond to it, and iterating,” OpenAI said.
By directly integrating GPT‑Red into the training process of its production models, OpenAI said GPT‑5.6 Sol is its most robust model to prompt injections to date, achieving 6x fewer failures against direct prompt injection benchmark compared to GPT-5.5, its frontier model from four months before.
Some of the sample prompt-injected conversations tested as part of the process include –
- Internal directory exfiltration
- Fraudulent payment instructions
- Amazon Web Services (AWS) credential exfiltration
- Disabling two-factor authentication (2FA)
- Credentials file upload
- External script injection
- API key forwarding
- Malicious scraper scripts
“GPT‑Red is trained using self-play reinforcement learning, where the model and a collection of diverse defender LLMs are trained simultaneously on a broad set of red-teaming scenarios,” OpenAI explained. “GPT‑Red is rewarded for eliciting a valid failure, such as a successful prompt injection, while the defender models are rewarded for resisting the attack and completing their original tasks.”
This also means that as the defender models get more robust, the red-teaming model will have to go back to the drawing board to discover more potent and diverse attack methods to defeat those guardrails. Specifically, GPT-Red has been found to generate successful attacks against GPT‑5.1 in more scenarios than human red-teamers when it comes to indirect prompt injections.
OpenAI further made it a point to emphasize that GPT‑Red is kept separate from the other models so that the malicious capabilities built into it do not reach bad actors who are constantly looking at various ways to bypass a model’s ethical and safety measures.
In one real-world test, OpenAI aimed GPT-Red at an AI-based vending machine built by Andon Labs. After practicing in simulation, the model targeted the autonomous agent and met all three of its goals: lowering the price of an expensive item to the minimum allowed price of $0.50, ordering a new $100 item for that same amount, and canceling another customer’s order. Following responsible disclosure, fresh safeguards are being tested, it added.
A second case study involved using GPT-Red to attack a Codex command-line agent, based on GPT-5.4 mini, across 10 held-out data-exfiltration tasks, causing sensitive data to be transmitted in more cases than a prompted GPT-5.5 baseline.
An early version of the model has also uncovered a novel class of direct prompt injection attacks known as Fake Chain-of-Thought (CoT) attacks, which achieved success rates north of 95% on GPT‑5.1 but are now below 10% for GPT‑5.6 Sol.
“Similarly, several of our indirect prompt injection benchmarks that target attacks in developer tools and browsing have been saturated by our latest model (>97% accuracy),” OpenAI said.
“Robustness to GPT‑Red itself has also improved substantially. On a broad set of robustness environments, GPT‑Red’s attack success rates have dropped monotonically over time. With our latest model release, GPT‑5.6 Sol fails on only 0.05% of GPT‑Red’s direct prompt injections.”
The disclosure comes as the company said an audit of SWE-Bench Pro found that about 30% of tasks are broken, retracting its previous recommendation to adopt the benchmark for measuring frontier coding capabilities. Earlier this February, OpenAI said it was moving away from SWE-bench Verified due to fundamental design and contamination issues.
“We find evidence of breaking issues in a significant portion of the dataset,” OpenAI said. “Our datapoint analysis pipeline flagged 200 (27.4%) broken tasks, while the human annotation campaign identified 249 (34.1%). Ultimately, an eval should provide meaningful signal through benchmarks that are hard to game, easy to trust, and genuinely reflective of model capability or alignment.”