This Defense Company Made AI Agents That Blow Things Up

Scout AI, a Silicon Valley startup, is adapting large AI models to create agents capable of physical destruction, unlike typical AI focused on digital tasks. In a recent demonstration at a California military base, their AI successfully directed an off-road vehicle and two drones to locate and destroy a target truck using an explosive charge.

CEO Colby Adcock stated the goal is to transition generalized foundation models into warfighter agents. The system uses a large model (Fury Orchestrator) to interpret high-level commands and issue orders to smaller models controlling the physical assets. This approach aims for greater autonomy than legacy systems, allowing agents to replan based on real-time information and commander intent.

While experts like Michael Horowitz acknowledge the importance of pushing AI integration in defense, he cautions that impressive demonstrations do not equate to military-grade reliability or cybersecurity, especially given the known unpredictability of large language models. Critics and ethicists also raise concerns about the complexities and ethical risks of granting AI lethal autonomy, though Scout AI claims its technology adheres to US military rules of engagement and international norms.

(Source：WIRED)