How OpenAI’s Performance Factory keeps ChatGPT fast
Summary
OpenAI has developed the Performance Factory, an autonomous system that leverages Codex and its models to automate performance engineering for ChatGPT. The system evolved from manual investigations using tools like profilers, to Codex as a partner assisting engineers, to reusable skills such as perf-forensics, and finally to automated workflows. The first workflow, the Regression Workflow, handles performance alerts: an agent investigates the slowdown, proposes a fix, an engineer reviews and deploys, and another agent verifies recovery. In over three months, it tracked over 1,700 investigations and root-caused over 600 cases. The Performance Factory consists of a task tracker with configurable workflows, a dispatch loop, a cloud API for running Codex, and subscriptions for events. Key lessons include protecting engineers’ attention by summarizing and ranking findings, evaluating agent changes through backtesting on past incidents, and using agents to self-repair recurring failures. Future work includes proactive monitoring and broader applications of the agent factory pattern to continuously optimize services.
(Source:OpenAI)