What If Automating AI R&D Triggers an Intelligence Explosion? | The Foundation for American Innovation

AI systems now write most of the code inside the companies that build them, and more of the AI research and development (R&D) pipeline is being automated, raising the possibility of an "intelligence explosion" in which years of advances are compressed into months or less. Preliminary evidence suggests this could happen. The authors assess this evidence, analyze the potential impacts of an intelligence explosion, and propose policy responses. AI systems are on track to automate most AI R&D work within a few years, and possibly all of it. If this triggers an intelligence explosion, it could dramatically bring forward AI's benefits, but also pose extreme risks: capabilities growth could accelerate far beyond what society can keep up with, humanity could lose control over superhuman AI systems, and checks on power within and between states, companies, and branches of government could be severely eroded. Although much uncertainty remains, the high stakes warrant serious further attention. Policymakers should urgently obtain more visibility into the automation of AI R&D, develop ways to steer and constrain an intelligence explosion, and prepare society to adapt to its impacts.

(Source：Thefai)