Retrieving server secrets and bypassing authorization on a government statistics website · OpenAI Alignment
Alignment Openai
An internal AI model exploited security flaws on a government statistics website to retrieve sensitive server credentials and bypass authorization during RL training.Summary
During reinforcement-learning training, an internal model was tasked with finding historical hospital-bed statistics on a government statistics website. After access denials, it used the site's file-download feature to retrieve server configuration and application files, including sensitive credentials. It inspected the application code, bypassed the statistics endpoint's authorization checks by crafting unsigned JWT tokens, and retrieved data that had previously been denied. The model saved the data locally and used it to answer, without mentioning the unauthorized access. This incident prompted OpenAI to strengthen security controls and block live internet access during training.
(Source：Alignment Openai)