Artificial intelligence can detect and interpret social features between people from images and videos almost as reliably as humans, according to new a study from the University of Turku in Finland.
People are constantly making quick evaluations about each other's behaviour and interactions. The latest AI models, such as the large language model ChatGPT developed by OpenAI, can describe what is happening in images or videos. However, it has not been clear whether AI's interpretive capabilities are limited to easily recognisable details or whether it can also interpret complex social information.
Researchers at the Turku PET Centre in Finland studied how accurately the popular language model ChatGPT can assess social interaction. The model was asked to evaluate 138 different social features from videos and pictures. The features described a wide range of social traits such as facial expressions, body movements or characteristics of social interaction, such as co-operation or hostility. The researchers compared the evaluations made by AI with more than 2,000 similar evaluations made by humans.
The research results showed that the evaluations provided by ChatGPT were very close to those made by humans. AI's evaluations were even more consistent than those made by a single person.
“Since ChatGPT's assessment of social features were on average more consistent than those of an individual participant, its evaluations could be trusted even more than those made by a single person. However, the evaluations of several people together are still more accurate than those of artificial intelligence,” says Postdoctoral Researcher Severi Santavirta from the University of Turku.
Artificial intelligence can boost research in neuroscience
The researchers used AI and human participants' evaluations of social situations to model the brain networks of social perception using functional brain imaging in the second phase of the study. Before researchers can look at what happens in the human brain when people watch videos or pictures, the social situations they depict need to be assessed. This is where AI proved to be a useful tool.
“The results were strikingly similar when we mapped the brain networks of social perception based on either ChatGPT or people's social evaluations," says Santavirta.
Researchers say this suggests that AI can be a practical tool for large-scale and laborious neuroscience experiments, where, for example, interpreting video footage during brain imaging would require significant human effort. AI can automate this process, thereby reducing the cost of data processing and significantly speeding up research.
“Collecting human evaluations required the efforts of more than 2,000 participants and a total of more than 10,000 work hours, while ChatGPT produced the same evaluations in just a few hours,” Santavirta summarises.
Practical applications from healthcare to marketing
While the researchers focused on the benefits of AI for brain imaging research, the results suggest that AI could also be used for a wide range of other practical applications.
The automatic evaluation of social situations by AI from video footage could help doctors and nurses, for example, to monitor patients' well-being. Furthermore, AI could evaluate the likely reception of audiovisual marketing by the target audience or predict abnormal situations from security camera videos.
“The AI does not get tired like a human, but can monitor situations around the clock. In the future, the monitoring of increasingly complex situations can probably be left to artificial intelligence, allowing humans to focus on confirming the most important observations,” Santavirta says.