Although music is considered a purely auditory experience, visual cues can unconsciously influence performance judgements—a phenomenon known as the sight-over-sound effect. To explore how one’s musical expertise can affect this bias, researchers from Japan conducted a controlled study using recordings from Japanese high school brass band competitions. Their results reveal that the effect predominantly appears in musicians without genre-specific experience, highlighting how specialized training can shape multisensory evaluation in music perception and judgement.
When we listen to a symphony, a jazz performance, or a pop song, we naturally tend to assume that our ears are the primary judges of quality. For many people, music is primarily an auditory experience, and they instinctively believe that the true essence of a musical performance lies in factors such as its acoustic fidelity, proper intonation, or rhythmic precision. Research has, however, revealed an intriguing phenomenon that challenges this assumption. Dubbed as the “sight-over-sound effect,” this phenomenon suggests that visual aspects of a performer—from their stage presence to subtle body language cues—can unconsciously sway our judgment, sometimes even more powerfully than the music itself.
Despite its fascinating implications, the replicability of the sight-over-sound effect has remained debatable across musical styles and performance settings. Previous research has struggled with accurately assessing visual influences, experiencing challenges with inconsistent filming angles, musical pieces, and definitions of musical experience among evaluators. Consequently, the understanding of the sight-over-sound effect and its extent remains unclear.
Against this backdrop, a research team led by Associate Professor Shinya Fujii, Director of the Neurosciences and Music Laboratory at Keio University and the Research Center for Music Science at Keio University Global Research Institute, Japan, conducted a comprehensive study to investigate the replicability of this effect. The study was co-authored by Mr. Tomohiro Samma from the Graduate School of Media and Governance at Keio University and
published online in the PLOS One journal on April 29, 2025. The team meticulously investigated the replicability of the sight-over-sound effect, with a special focus on the role of the evaluators’ specific musical background.
To address the limitations of prior research, the researchers designed an experiment using recordings from Japanese high school brass band competitions. This allowed for significant control over the experimental stimuli. All selected bands had received gold awards, minimizing differences in skill level. Most importantly, all performances within a comparison set featured the same musical piece filmed with consistent camera angles. 301 adults were included in the study, who were categorized into three groups: brass band musicians (BMs) with direct experience in the genre, non-brass band musicians (NBMs) with general musical experience but no brass band background, and non-musicians (NMs) with no formal musical training. These groups evaluated brass band performances under audio-only, visual-only, or audio-visual conditions, selecting the band they believed would achieve the best competition results.
The study’s results revealed a markedly nuanced picture of the sight-over-sound effect. When analyzing the overall participant sample, the researchers found no significant evidence of the effect, suggesting that a careful control over visual elements and musical pieces could indeed reduce visual dominance. However, detailed subgroup analyses suggested a dependency on the evaluator’s musical experience. Specifically, the sight-over-sound effect was observed in NBMs, who showed significantly higher accuracy in identifying winners from visual-only information. In contrast, the effect was absent in BMs, who demonstrated superior accuracy in the audio-only condition, indicating their ability to make precise judgments based solely on sound. The NM group did not exhibit the sight-over-sound effect either.
These findings carry significant implications for our understanding of multisensory integration and the practical aspects of music and music competitions. Firstly, the absence of the sight-over-sound effect in the overall sample highlights the importance of methodological rigor in future studies. More importantly, varying results across the three groups underscore that specialized musical training can profoundly shape how individuals process and prioritize sensory information. “
Our study found that the sight-over-sound effect was observed only in NBMs, indicating its dependence on the evaluator’s specific musical experience, while BMs' auditory training potentially mitigating visual influence," explains Mr. Samma.
Additionally, the results are also relevant in social psychology and cognitive science in general, since they suggest that expertise in a given domain can alter one’s weighting of sensory inputs. For instance, visual dominance observed in the NBM group suggests that while they possess musical skills, they may still rely on visual cues for evaluation if they lack experience in a given genre. “
By shedding light on the underexplored aspect of how evaluators’ musical experience influences the evaluation process in multisensory integration, this work has significant implications for various real-world aspects, such as music education, performance, and competition judging,” concludes Dr. Fujii.
Further work on this intriguing topic will hopefully help us improve music education and make competitions fairer.
***
Reference
Title of original paper: Sight-over-sound effect depends on interaction between evaluators’ musical experience and auditory-visual integration: An examination using Japanese brass band competition recordings
Journal:
PLOS One
DOI:
https://doi.org/10.1371/journal.pone.0321442
About Keio University Global Research Institute (KGRI), Japan
The Keio University Global Research Institute (KGRI) was established in November 2016 as a research organization to bridge faculties and graduate schools across the university. KGRI aims to promote interdisciplinary and international collaborative research that goes beyond the boundaries of singular academic disciplines and international borders. It also aims to share research outcomes both in Japan and worldwide, further promoting engagement in joint research. To enhance those activities above, in 2022, Keio University set its goal of becoming a “Research university that forges the common sense of the future”.
Website:
https://www.kgri.keio.ac.jp/en/index.html
About Associate Professor Shinya Fujii from Keio University
Dr. Shinya Fujii obtained MS and Ph.D. degrees in Human and Environmental Studies from Kyoto University in 2007 and 2010, respectively. He currently serves as an Associate Professor at the Faculty of Environment and Information Studies, Keio University. He is also the Director of the Neurosciences and Music Laboratory at Keio University and the Research Center for Music Science at Keio University Global Research Institute. Dr. Fujii specializes in perception neuroscience and its relationship with music. He has over 75 papers credited to his name.
https://neuromusic.sfc.keio.ac.jp/
https://www.kgri.keio.ac.jp/en/project/research-centers/2024/A24-28.html
https://www.k-ris.keio.ac.jp/html/100012635_en.html
https://researchmap.jp/7000025148?lang=en
About Tomohiro Samma from Keio University
Tomohiro Samma is a Ph.D. student at the Graduate School of Media and Governance, Keio University. He specializes in musical neuroscience and has three papers credited to his name. He is a member of the Society for Music Perception and Cognition and the Society for the Neural Control of Movement.
https://researchmap.jp/tsamma?lang=en
About Kazuaki Honda from Communication Science Laboratories, NTT, Inc.
Dr. Kazuaki Honda obtained a Ph.D. degree from the Graduate School of Media and Governance, Keio University, in 2024. He is a research scientist at NTT Communication Science Labs. Dr. Honda specializes in perceptual and cognitive neuroscience and its relationship with music.
https://www.rd.ntt/e/cs/team_project/human/embodied_intelligence/
Funding information
This work was supported by Taikichiro Mori Memorial Research Grants, the Japan Society for the Promotion of Science Grant-in-aid (16K16483 and 20H04092), Japan Science and Technology Corporation (JST COI-NEXT Grant: JPMJPF2203), Keio University Academic Development Funds, and Toyama Memorial Music Research Foundation.