Combatting adversarial distillation
Summary
OpenAI describes adversarial distillation, where bad actors use a frontier model's outputs without authorization to train another model with similar capabilities but without comparable safeguards, posing risks especially in dual-use domains like cybersecurity and biology. The company details its defense strategy, which combines adversarial testing, model training, classifiers, and API-level protections with investigations into coordinated extraction campaigns. A key focus is protecting the model's chain-of-thought reasoning, as exact reasoning traces can transfer more capability than summaries. In internal evaluations, updated safeguards in GPT-6.2 Astra reduced the average attack success rate for extracting protected reasoning from 54.0% against GPT-6 Astra to 0.2%. OpenAI also enforces against networks of accounts conducting distillation campaigns, shares indicators of compromise with other providers, and supports legislation and export controls to address national security risks. The article concludes with recommendations for other developers and platforms to strengthen distillation defenses industry-wide.
(Source:OpenAI)