Eye movements are central to human perception and cognition. During naturalistic activities such as movie watching, gaze patterns reflect changes in attention, engagement, and comprehension. However, many fMRI datasets lack simultaneous eye-tracking recordings, making it difficult to investigate how eye movements relate to brain activity.
Recently, deep learning methods such as DeepMReye have made it possible to estimate gaze position directly from fMRI signals of the eyes, which are captured in most brain scans. This offers a way to recover gaze information from existing brain scans. Yet, their accuracy across different datasets—especially without additional training—remains uncertain.
To examine this possibility, researchers from the New Jersey Institute of Technology applied a pre-trained DeepMReye model to three independent public movie-watching fMRI datasets covering children and adults aged 3 to 51. They validated the model’s predicted gaze against camera-based eye-tracking measurements in the one dataset where they were available, tested the consistency of predictions in the other two, and then used the predicted eye movements to identify related brain activity.
The results showed that individual-level predictions had modest accuracy (correlations of about 0.24–0.37 with measured gaze), while group-averaged predictions closely matched actual gaze patterns (about 0.73–0.84). Importantly, brain activation maps based on group-averaged predictions revealed widespread activity in established eye-movement control regions, including the frontal and parietal eye fields. In contrast, individual-level predictions produced more restricted activation patterns.
These findings suggest that group-averaged fMRI-based gaze estimates can provide a practical approach for studying shared viewing behaviors and eye-movement-related brain activity in datasets without eye-tracking recordings. However, the authors note that precise individual-level gaze measurements still require further methodological improvements.
Many large fMRI datasets were collected without eye tracking, which limits what we can learn from them, said Xin Di, the study’s corresponding author. He added that their results show that the scans themselves contain enough information to recover how a group of viewers moved their eyes, though not yet reliably for each individual.
The work titled “
fMRI-based prediction of eye gaze during naturalistic movie viewing reveals eye-movement-related brain activity” was published in
Psychoradiology (published on June 26, 2026).
DOI:10.1093/psyrad/kkag026