The importance of audio descriptors in automatic soccer highlights generation

Abstract

Automatic generation of sports highlights from recorded audiovisual content has been object of great interest in recent years. The problem is indeed important in the production of second and third division leagues highlights videos where the quantity of raw material is significant and does not contain manual annotations. Many approaches are mostly based on the analysis of the video and disregard the important information provided by the audio track. In this paper, a new approach that combines audio and video descriptors for automatic soccer highlights generation is proposed. The approach is based on the segmentation of the video contents into shots that are further analyzed in order to determine its relevance and interest. These video-shots are scored taking into account the fusion between different audio and video features. The paper is mainly focused to emphasize the importance of audio detectors that play a key role in the analysis and scoring of the video-shots. Specifically, a new algorithm for referee's whistle detection is proposed. The algorithm has been proven to be very robust and efficiently discriminates professional whistles against other types of noises such as public cheering-up, music instruments, etc. Several results have been produced using real soccer video sequences that prove the validity of the proposed audio and video fusion scheme.Peer ReviewedPostprint (published version

[Papers] Multimodal Important Scene Detection in Far-view Soccer Videos Based on Single Deep Neural Architecture

“…It is known that the frequency band of sounds of the whistle blown by the referee is in the range of 3.5-4.5 kHz [36]. Thus, when a whistle is blown, it is considered that there is an energy peak in the frequency band of 3.5-4.5 kHz.…”

Multidimensional projections for the visual exploration of multimedia data

“…End-user systems: Apart from the above more advanced visualization systems, which aim to provide tactical and statistical information to expert users, several other systems have been designed for the casual user (typical soccer fan), such as video summarization (TAVASSOLIPOUR;KARIMIAN;KASAEI, 2014;RAVENTOS et al, 2014;YOSHITAKA, 2014) and content retrieval systems(SULSER; GIANGRECO; SCHULDT, KOLEKAR, 2011). In this context, the main aims are to help casual users to save time while browsing match videos, by attracting attention to the main match events.…”

“…The main goal of video summarization is to detect typical events of interest (e.g. foul, goal, shot, corner, offside) (TAVASSOLIPOUR; KARIMIAN; KASAEI, 2014), or detect the main match events, also known as highlights (RAVENTOS et al, 2014;YOSHITAKA, 2014). For example, thevideo summarization in (TAVASSOLIPOUR; KARIMIAN; KASAEI, 2014) usesBayesian networksto classify and detect, through " play-break" semantic units, seven different event types: goal, card, goal attempt, corner, offside, foul and non-highlight.…”

“…This idea was extended to capture not only game highlights, but also competitionintensive scenes and emotional events, which are presented as frame sequences YOSHITAKA, 2014). Soccer highlights can be automatically generated from audio and video descriptors (RAVENTOS et al, 2014). After segmenting the video into shots, audio detectors including an algorithm for referee's whistle were used to create the highlights.…”

“…Different kinds of insights and information can be identified using soccer video analysis(OSKOUIE; ALIPOUR; EFTEKHARI-MOGHADAM, 2014). Users who wish to see only important or interesting events, also known as highlights, are best served by match summarization systems(TAVASSOLIPOUR; KARIMIAN;KASAEI, 2014;RAVENTOS et al, 2014;YOSHITAKA, 2014). Users who want to see the team strategy, by e.g.…”