Introducing the Multimodal Knowledge Agent benchmark

Google DeepMind, in partnership with Artificial Analysis, has launched the Multimodal Knowledge Agent (MKA) benchmark, which evaluates how AI agents reason across visually complex files to solve real-world knowledge work tasks. Developed with domain experts across 10 verticals and 83 occupations, the benchmark consists of 200 multi-step tasks that take professionals three to eight hours to complete, paired with expert-written rubrics weighted 75% on factual accuracy and 25% on visual design. An agentic autorater built on a non-Google model grades each deliverable, agreeing with human experts 96% of the time. The evaluation set is kept private to prevent leakage, while 10 public tasks are released on Hugging Face. Gemini 4 Carbon achieves second place in quality while completing tasks at 6x lower cost on average compared to the top-scoring model, placing it on the pareto frontier of quality and price.

(Source：Gemini)