Two web experiments in Japan found that criticism of the media appeared stronger when people were asked directly than when the same view was measured indirectly

Osaka, Japan - In recent years, strong criticism of the mass media has become widespread online in Japan. Against this backdrop, researchers at the University of Osaka examined whether direct survey questions may make negative attitudes toward the news media appear stronger than they actually are. In two web-based randomized experiments, agreement with the statement “the mass media are harmful to society” was higher when respondents were asked directly than when the same view was measured indirectly using a list experiment.

Across the two studies combined, 45.1% of respondents agreed with the statement in the direct-question condition. By contrast, the list experiment yielded an estimated agreement rate of 29.7%. The gap of 15.4 percentage points was statistically significant, and the same directional pattern appeared in both studies.

The findings suggest that, in social settings where criticizing the media is easy to express, direct questions may capture not only private opinion but also socially shaped response tendencies. In other words, negative attitudes toward the media may sometimes be overstated rather than hidden.

To test this, Professor Asako Miura of the Graduate School of Human Sciences at the University of Osaka conducted two online survey experiments in Japan. Some respondents were asked directly whether they thought the mass media were harmful to society, while others answered a list experiment designed to reduce pressure to state that view explicitly.

The study does not suggest that criticism of the media is insincere. Rather, it shows that the way a question is asked can affect how strongly such criticism appears in survey results. The paper also notes that both studies used nonprobability online samples, so the findings should not be treated as a direct estimate of opinion in Japan as a whole.
Professor Miura commented: “Seeing consistently high levels of media distrust in questionnaire surveys, I wondered whether the way we asked the question might be shaping the results.”
###

The article, “When Negative Media Attitudes Are Overreported: Comparing Direct and List-Experiment Measures in Japan,” will be published in International Journal of Public Opinion Research at DOI: https://doi.org/10.1093/ijpor/edag013

About The University of Osaka
The University of Osaka was founded in 1931 as one of the seven imperial universities of Japan and is now one of Japan's leading comprehensive universities with a broad disciplinary spectrum. This strength is coupled with a singular drive for innovation that extends throughout the scientific process, from fundamental research to the creation of applied technology with positive economic impacts. Its commitment to innovation has been recognized in Japan and around the world. Now, The University of Osaka is leveraging its role as a Designated National University Corporation selected by the Ministry of Education, Culture, Sports, Science and Technology to contribute to innovation for human welfare, sustainable development of society, and social transformation.
Website: https://resou.osaka-u.ac.jp/en