KTU researchers enhance live football broadcasts by eliminating cameramen distractions with AI
en-GBde-DEes-ESfr-FR

KTU researchers enhance live football broadcasts by eliminating cameramen distractions with AI


While the sports industry continually improves spectators’ viewing experience at home, some problems remain unsolved. One such issue for football fans is cameramen accidentally appearing in each other’s shots on live broadcasts. These occurrences not only detract from critical game moments but potentially lead to revenue losses for broadcasters due to viewer dissatisfaction. To address this, researchers at Kaunas University of Technology (KTU) have developed an end-to-end system to enhance the viewing experience by eliminating visual distractions caused by overlapping camera angles.

“Our new invention is an algorithm adapted to detect video operators,” says KTU professor Rytis Maskeliūnas, one of the creators of the innovation.

Serhii Postupaiev, another members of the research team, highlights that the presence of cameramen in the frame is a frequent issue in football broadcasts due to the complex nature of live sports coverage and the numerous cameras around the stadium.

“The number of camera points in prestigious tournaments can start from nine and numerous overlapping views that contribute to the visual distraction issues. These issues severely constrain the cameramen team, as they constantly have to film the game while avoiding capturing each other, potentially leading to a loss of context in some game moments or making the broadcast less dynamic and immersive,” explains Postupaiev.

Eliminating visual distractions

To address this problem and remove unwanted objects during a live broadcast, KTU scientists designed and implemented an end-to-end system.

For its functioning, the YOLOv8 model, a state-of-the-art object detection system known for its speed and accuracy, was employed. YOLOv8, which stands for “You Only Look Once”, can detect and classify objects in images in a single pass, making it ideal for real-time events such as live football broadcasts.

“It works by dividing the image into a grid and predicting bounding boxes, class probabilities, and segmentation polygons for each grid cell. This enables it to identify and segment cameramen,” says Serhii Postupaiev, who recently graduated from KTU with a Master’s degree in Artificial Intelligence in Computer Science.

To train the YOLOv8 model to accurately detect and segment cameramen during football matches, a dataset had to be created.

“I created this dataset to include a diverse range of cameramen with different sizes, shapes, and types of equipment, captured under various conditions and at different stages of the game. Now YOLOv8 uses this dataset to identify where cameramen are in the video frames,” adds Postupaiev.

As the inventor explains, this process was needed to create the foundation for the actual removal of the operators. For this purpose, video inpainting technology was used.

“The term inpainting in deep learning refers to the process of reconstructing lost or deteriorated parts of images and videos. Specifically, in this case, it is used for removing cameramen from football video broadcasts,” says Postupaiev.

Artificial intelligence (AI) and computer vision-based technology analyses the video frames to detect unwanted objects such as cameramen and fills the removed areas with relevant background details. The modified frames are then streamed back to viewers, ensuring a more immersive and professional broadcast.

Maskeliūnas adds that on television servers, this algorithm could process the recorded image before it is broadcast on air with a delay of a few seconds from the actual captured moment, which is still considered a live broadcast. He believes that, as the equipment improves, AI will fill this time gap perfectly.

Shifting the focus from simply capturing the action

With this new technology, watching football matches at home will be significantly improved. One of them is a smoother viewing experience.

“The broadcast will feel more polished and professional without disruptions caused by cameramen appearing where they shouldn’t. This improvement will reduce the number of cases where important moments of the game are missed due to distracting shots,” highlights Postupaiev, who received his Master’s degree with this project.

According to Postupaiev, further research in this area could usher in a new era of sports broadcasting, shifting the focus from simply capturing the action to creating a fully immersive and uninterrupted viewing experience.

“By implementing cameramen inpainting, broadcast companies can explore innovative camera angles, perspectives, and effects, bringing games to life in new and exciting ways,” he says.
Additionally, cameramen inpainting can extend beyond live broadcasts to enhance pre- and post-match analyses, offline highlight reel processing, and restoration of archival footage.
“This could even breathe new life into old recordings of classic matches,” adds a KTU graduate.

Invention is not limited to football – it can be applied to other sports with similar broadcasting challenges. Dynamic sports like futsal and basketball, which require immersive broadcasts, can also benefit from this technology.

“This is yet another illustration of what modern AI applications can do. We often hear about medical applications, but here we have a consumer-oriented approach to editing images we don’t like. In the future, such technology will be able, for example, to remove advertisements or replace them with other ones thus constantly updating the content with a level of precision that the human eye will not notice,” says KTU Faculty of Informatics professor Maskeliūnas.

The paper Real-time camera operator segmentation with YOLOv8 in football video broadcasts can be found here.
Postupaiev S, Damaševičius R, Maskeliūnas R. Real-Time Camera Operator Segmentation with YOLOv8 in Football Video Broadcasts. AI. 2024; 5(2):842-872. https://doi.org/10.3390/ai5020042
Attached files
  • Cameramen video inpainting base use-case
  • erhii20ostupaiev.jpg
  • KTU Faculty of Informatics professor Rytis Maskeliūnas
Regions: Europe, Lithuania
Keywords: Applied science, Artificial Intelligence

Testimonials

For well over a decade, in my capacity as a researcher, broadcaster, and producer, I have relied heavily on Alphagalileo.
All of my work trips have been planned around stories that I've found on this site.
The under embargo section allows us to plan ahead and the news releases enable us to find key experts.
Going through the tailored daily updates is the best way to start the day. It's such a critical service for me and many of my colleagues.
Koula Bouloukos, Senior manager, Editorial & Production Underknown
We have used AlphaGalileo since its foundation but frankly we need it more than ever now to ensure our research news is heard across Europe, Asia and North America. As one of the UK’s leading research universities we want to continue to work with other outstanding researchers in Europe. AlphaGalileo helps us to continue to bring our research story to them and the rest of the world.
Peter Dunn, Director of Press and Media Relations at the University of Warwick
AlphaGalileo has helped us more than double our reach at SciDev.Net. The service has enabled our journalists around the world to reach the mainstream media with articles about the impact of science on people in low- and middle-income countries, leading to big increases in the number of SciDev.Net articles that have been republished.
Ben Deighton, SciDevNet

We Work Closely With...


  • BBC
  • The Times
  • National Geographic
  • The University of Edinburgh
  • University of Cambridge
Copyright 2024 by AlphaGalileo Terms Of Use Privacy Statement