Reading Eye Movements from Brain Scans: Can Deep Learning Predict How We Watch Movies?
en-GBde-DEes-ESfr-FR

Reading Eye Movements from Brain Scans: Can Deep Learning Predict How We Watch Movies?

18/11/2026 HEP Journals

Eye movements are central to human perception and cognition. During naturalistic activities such as movie watching, gaze patterns reflect changes in attention, engagement, and comprehension. However, many fMRI datasets lack simultaneous eye-tracking recordings, making it difficult to investigate how eye movements relate to brain activity.

Recently, deep learning methods such as DeepMReye have made it possible to estimate gaze position directly from fMRI signals of the eyes, which are captured in most brain scans. This offers a way to recover gaze information from existing brain scans. Yet, their accuracy across different datasets—especially without additional training—remains uncertain.

To examine this possibility, researchers from the New Jersey Institute of Technology applied a pre-trained DeepMReye model to three independent public movie-watching fMRI datasets covering children and adults aged 3 to 51. They validated the model’s predicted gaze against camera-based eye-tracking measurements in the one dataset where they were available, tested the consistency of predictions in the other two, and then used the predicted eye movements to identify related brain activity.

The results showed that individual-level predictions had modest accuracy (correlations of about 0.24–0.37 with measured gaze), while group-averaged predictions closely matched actual gaze patterns (about 0.73–0.84). Importantly, brain activation maps based on group-averaged predictions revealed widespread activity in established eye-movement control regions, including the frontal and parietal eye fields. In contrast, individual-level predictions produced more restricted activation patterns.

These findings suggest that group-averaged fMRI-based gaze estimates can provide a practical approach for studying shared viewing behaviors and eye-movement-related brain activity in datasets without eye-tracking recordings. However, the authors note that precise individual-level gaze measurements still require further methodological improvements.

Many large fMRI datasets were collected without eye tracking, which limits what we can learn from them, said Xin Di, the study’s corresponding author. He added that their results show that the scans themselves contain enough information to recover how a group of viewers moved their eyes, though not yet reliably for each individual.

The work titled “fMRI-based prediction of eye gaze during naturalistic movie viewing reveals eye-movement-related brain activity” was published in Psychoradiology (published on June 26, 2026).
DOI:10.1093/psyrad/kkag026
Attached files
  • Image: Validation of DeepMReye gaze predictions against ground-truth eye-tracking data.
18/11/2026 HEP Journals
Regions: Asia, China
Keywords: Science, Life Sciences

Disclaimer: AlphaGalileo is not responsible for the accuracy of content posted to AlphaGalileo by contributing institutions or for the use of any information through the AlphaGalileo system.

Testimonials

For well over a decade, in my capacity as a researcher, broadcaster, and producer, I have relied heavily on Alphagalileo.
All of my work trips have been planned around stories that I've found on this site.
The under embargo section allows us to plan ahead and the news releases enable us to find key experts.
Going through the tailored daily updates is the best way to start the day. It's such a critical service for me and many of my colleagues.
Koula Bouloukos, Senior manager, Editorial & Production Underknown
We have used AlphaGalileo since its foundation but frankly we need it more than ever now to ensure our research news is heard across Europe, Asia and North America. As one of the UK’s leading research universities we want to continue to work with other outstanding researchers in Europe. AlphaGalileo helps us to continue to bring our research story to them and the rest of the world.
Peter Dunn, Director of Press and Media Relations at the University of Warwick
AlphaGalileo has helped us more than double our reach at SciDev.Net. The service has enabled our journalists around the world to reach the mainstream media with articles about the impact of science on people in low- and middle-income countries, leading to big increases in the number of SciDev.Net articles that have been republished.
Ben Deighton, SciDevNet

We Work Closely With...


  • The Research Council of Norway
  • SciDevNet
  • Swiss National Science Foundation
  • iesResearch
Copyright 2026 by AlphaGalileo Terms Of Use Privacy Statement