An AI “mind-reading” tool can reconstruct what you’re looking at based on a brain scan

A new AI tool developed by Michal Irani and her team at the Weizmann Institute of Science in Rehovot, Israel, can reconstruct what a person is looking at by analyzing their brain scans.

The tool uses functional magnetic resonance imaging (fMRI) scans, which track blood flow in the brain to identify active areas.

While traditional fMRI scans highlight brain activity in voxels (3D pixels) covering about 3 cubic millimeters, Irani’s team used higher-resolution scans with voxels of 1 cubic millimeter.

The AI model consists of two parts: a decoder that predicts the structure and content of an image from brain activity, and an encoder that does the reverse—predicting brain activity based on an image.

The tool was trained on data from eight people who viewed around 9,000 images each in an fMRI scanner.

The reconstructed images are compared to the originals, showing remarkable accuracy in recreating details like colors, shapes, and positions.