Adaptive extraction of highlights from a sport video based on excitement modeling

Abstract

Abstract-This paper addresses the challenge of automatically extracting the highlights from sports TV broadcasts. In particular, we are interested in finding a generic method of highlights extraction, which does not require the development of models for the events that are thought to be interpreted by the users as highlights. Instead, we search for highlights in those video segments that are expected to excite the users most. It is namely realistic to assume that a highlighting event induces a steady increase in a user's excitement, as compared to other, less interesting events. We mimic the expected variations in a user's excitement by observing the temporal behavior of selected audiovisual low-level features and the editing scheme of a video. Relations between this noncontent information and the evoked excitement are drawn partly from psychophysiological research and partly from analyzing the live-video directing practice. The expected variations in a user's excitement are represented by the excitement time curve, which is, subsequently, filtered in an adaptive way to extract the highlights in the prespecified total length and in view of the preferences regarding the highlights strength: extraction can namely be performed with variable sensitivity to capture few "strong" highlights or more "less strong" ones. We evaluate and discuss the performance of our method on the case study of soccer TV broadcasts.Index Terms-Affective video content analysis, video abstraction, video content modeling, video content pruning, video highlights extraction.

Analysing user physiological responses for affective video summarisation

“…For example, Shao et al [52] summarise music videos automatically by using an adaptive clustering algorithm, and music domain knowledge to analyse content in the music track, while detecting and clustering shots in the video track. In contrast, motion activity, cut density and sound energy have been used to produce content-based excitement curves [30] and affect curves that probabilistically infer how the user's affective state might be changed by the video content [31]. Coicca and Schettini [12] analyse image features, in terms of the level of difference between two consecutive frames, resulting in a frame by frame measure of visual complexity.…”

General Highlight Detection in Sport Videos

“…This methodology is relatively new in sports video analysis [3]. Ma et al [10] employ a series of psychological models on pre-attention, i.e.…”

“…With the increase of feature number, noise overwhelms actual attention peaks and thus fails highlight detection. Hanjalic et al [3] carefully choose three features to estimate the intensity of viewer reflection, including block motion vector, shot cut density and audio energy. The authors furthermore employ a 1-minute long low-pass Kaiser window filter to smooth these features as well as enhance the signal noise rate (SNR) of feature related attention [4].…”

“…Given that all highlights incur strong reflections among viewers, i.e. happiness or surprise, attention, the psychological measurement of human reflection, is proposed in [3] [10] as an efficient method to identify general highlights. Moreover, the estimation of attention intensity concerns few sports semantics.…”

Event Tactic Analysis Based on Broadcast Sports Video

“…In order to improve the robustness of event detection, multimodality based approaches were employed for semantic extraction in sports video. For example, audio/visual features were utilized for highlight extraction [6], [7], and audio/visual/textual features were utilized for event detection [5], [9].…”