Researchers at University College London's Sainsbury Wellcome Centre have reconstructed short videos purely from the brain activity of mice, recreating what the animals were watching without any input beyond their neural signals. The study, led by Dr. Joel Bauer with co-authors Troy Margrie and Claudia Clopath, was published September 16 in eLife.
The team used two-photon calcium imaging to track roughly 8,000 neurons in the primary visual cortex of 10 mice as they watched short video clips. A dynamic neural encoding model, built on techniques from the 2023 Sensorium Competition, was trained to link patterns of neural firing to what appeared on screen. When shown brain activity from a video the model had never seen before, it reconstructed a 10-second clip that closely resembled the original, pixel by pixel.
Dr. Bauer said the goal was "a better way of investigating how the brain interprets what we see," noting that the visual processing pipeline "skews and warps our representation" of the world. Reconstruction quality improved as more individual neurons were included, and the model showed only small timing differences between the real and rebuilt footage. The researchers say the approach could help scientists probe how different species perceive their surroundings and support future research into visual processing disorders.