Profiling Actions for Sport Video Summarization: An attention signal analysis

Abstract

Analyzing video content to produce summaries and extracting highlights has been challenging for decades. One of the biggest challenges for automatic sports video summarization is to produce summaries almost immediately after it ended, witnessing the course of the match while preserving emotions. Currently, in broadcast companies many human operators select which actions should belong to the summary based on multiple rules they have built upon their own experience using different sources of information. These rules define the different profiles of actions of interest that help the operator to generate better customized summaries. Most of these profiles do not directly rely on broadcast video content but rather exploit metadata describing the course of the match. In this paper, we show how the signals produced by the attention layer of a recurrent neural network can be seen as a learnt representation of these action profiles and provide a new tool to support operators' work. The results in soccer matches show the capacity of our approach to transfer knowledge between datasets from different broadcasting companies, from different leagues, and the ability of the attention layer to learn meaningful action profiles.

A Graph-Based Method for Soccer Action Spotting Using Unsupervised Player Classification

“…These operators not only rely on recorded broadcast videos, but also in event metadata coming from logs, GPS data, and video-tracking data [35]. This is a laborious task considering that a soccer match lasts around 90 minutes, but it can contain more than 1,500 events [39] and take about 8 hours to annotate them [17,48].…”