Reading Eye Movements from Brain Scans: Can Deep Learning Predict How We Watch Movies?
en-GBde-DEes-ESfr-FR

Reading Eye Movements from Brain Scans: Can Deep Learning Predict How We Watch Movies?

18.11.2026 HEP Journals

Eye movements are central to human perception and cognition. During naturalistic activities such as movie watching, gaze patterns reflect changes in attention, engagement, and comprehension. However, many fMRI datasets lack simultaneous eye-tracking recordings, making it difficult to investigate how eye movements relate to brain activity.

Recently, deep learning methods such as DeepMReye have made it possible to estimate gaze position directly from fMRI signals of the eyes, which are captured in most brain scans. This offers a way to recover gaze information from existing brain scans. Yet, their accuracy across different datasets—especially without additional training—remains uncertain.

To examine this possibility, researchers from the New Jersey Institute of Technology applied a pre-trained DeepMReye model to three independent public movie-watching fMRI datasets covering children and adults aged 3 to 51. They validated the model’s predicted gaze against camera-based eye-tracking measurements in the one dataset where they were available, tested the consistency of predictions in the other two, and then used the predicted eye movements to identify related brain activity.

The results showed that individual-level predictions had modest accuracy (correlations of about 0.24–0.37 with measured gaze), while group-averaged predictions closely matched actual gaze patterns (about 0.73–0.84). Importantly, brain activation maps based on group-averaged predictions revealed widespread activity in established eye-movement control regions, including the frontal and parietal eye fields. In contrast, individual-level predictions produced more restricted activation patterns.

These findings suggest that group-averaged fMRI-based gaze estimates can provide a practical approach for studying shared viewing behaviors and eye-movement-related brain activity in datasets without eye-tracking recordings. However, the authors note that precise individual-level gaze measurements still require further methodological improvements.

Many large fMRI datasets were collected without eye tracking, which limits what we can learn from them, said Xin Di, the study’s corresponding author. He added that their results show that the scans themselves contain enough information to recover how a group of viewers moved their eyes, though not yet reliably for each individual.

The work titled “fMRI-based prediction of eye gaze during naturalistic movie viewing reveals eye-movement-related brain activity” was published in Psychoradiology (published on June 26, 2026).
DOI:10.1093/psyrad/kkag026
Angehängte Dokumente
  • Image: Validation of DeepMReye gaze predictions against ground-truth eye-tracking data.
18.11.2026 HEP Journals
Regions: Asia, China
Keywords: Science, Life Sciences

Disclaimer: AlphaGalileo is not responsible for the accuracy of content posted to AlphaGalileo by contributing institutions or for the use of any information through the AlphaGalileo system.

Referenzen

We have used AlphaGalileo since its foundation but frankly we need it more than ever now to ensure our research news is heard across Europe, Asia and North America. As one of the UK’s leading research universities we want to continue to work with other outstanding researchers in Europe. AlphaGalileo helps us to continue to bring our research story to them and the rest of the world.
Peter Dunn, Director of Press and Media Relations at the University of Warwick
AlphaGalileo has helped us more than double our reach at SciDev.Net. The service has enabled our journalists around the world to reach the mainstream media with articles about the impact of science on people in low- and middle-income countries, leading to big increases in the number of SciDev.Net articles that have been republished.
Ben Deighton, SciDevNet
AlphaGalileo is a great source of global research news. I use it regularly.
Robert Lee Hotz, LA Times

Wir arbeiten eng zusammen mit...


  • The Research Council of Norway
  • SciDevNet
  • Swiss National Science Foundation
  • iesResearch
Copyright 2026 by DNN Corp Terms Of Use Privacy Statement