In a study, AI model OpenScholar synthesizes scientific research and cites sources as accurately as human experts

Researchers from the University of Washington (UW) and The Allen Institute for AI (Ai2) developed OpenScholar, an open-source AI model specifically designed to synthesize current scientific research while accurately citing sources, addressing the hallucination issues common in general-purpose models like GPT-4o, which fabricated 78-90% of citations in a prior study.

The team created ScholarQABench, the first large, multi-domain benchmark for evaluating research synthesis, using 3,000 queries and 250 expert-written answers. OpenScholar was trained on 45 million scientific papers and uses retrieval-augmented generation to incorporate emerging research. In tests, OpenScholar cited sources as accurately as human experts, and scientists preferred its responses over those written by subject experts 51% of the time.

Senior author Hannaneh Hajishirzi noted the high demand for this transparent, open-source system. When OpenScholar's citation methods were combined with GPT-4o, scientists preferred the AI-written answers 70% of the time, significantly higher than the 32% preference rate for GPT-4o alone. The project's code and demo are publicly available, and the team is already working on a follow-up model called DR Tulu.

(Source：UW News)