GPT-Red: Unlocking Self-Improvement for Robustness
OpenAI
OpenAI introduced GPT-Red, an automated red-teaming model that enhances AI robustness by generating adversarial training data to defend against prompt injection attacks.Summary
OpenAI has developed GPT-Red, an automated, internal-only red-teaming model designed to identify vulnerabilities and bolster the safety of future AI models. By utilizing self-play reinforcement learning, GPT-Red generates diverse and sophisticated adversarial attacks, allowing OpenAI to train models like GPT-5.6 to be significantly more resilient to prompt injections. This approach enables safety to scale alongside model capabilities, ensuring that performance is maintained while reducing the risk of malicious exploitation.
(Source：OpenAI)