New AI can reconstruct what you’re looking at by reading brain activity

Researchers developed Brain-IT, an AI system that reconstructs images a person is viewing by analyzing their fMRI brain activity. Trained on the Natural Scenes Dataset—thousands of images viewed by eight participants across 30–40 sessions each—the system separates semantic and structural features, using a diffusion model to generate reconstructions that preserve both content and spatial arrangement. It achieves competitive performance compared to leading methods on most metrics, and can adapt to a new person with only one hour of fMRI data, comparable to prior systems like MindEye2. The team also used an encoder-decoder loop to expand training data by predicting brain responses to unseen images, though these are not new human measurements. While the system still makes mistakes (e.g., a dog becomes a goat), it represents a significant advance in visual reconstruction. Proposed extensions include decoding dreams, aiding communication for paralyzed patients, and using EEG, but these remain speculative and raise ethical concerns about mental privacy.

(Source：The Brighter Side of News)