Multi-modal summarization of key events and top players in sports tournament videos

Tjondronegoro, Dian; Tao, Xiaohui; Sasongko, Johannes; Lau, Cher Han

doi:10.1109/wacv.2011.5711541

Cited by 23 publications

(20 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Erol et al (2003) aim to create important segments of a meeting recording based on audio, text and visual activity analysis. Tjondronegoro et al (2011) propose a way to summarize a sporting event by analyzing the textual information extracted from multiple resources and identifying the important content in a sport video. Evangelopoulos et al (2013) use an attention mechanism to detect salient events in a movie.…”

Section: Multi-modal Summarizationmentioning

confidence: 99%

“…The existing applications related to MMS include meeting record summarization (Erol et al, 2003;Gross et al, 2000), sport video summarization (Tjondronegoro et al, 2011;Hasan et al, 2013), movie summarization (Evangelopoulos et al, 2013;Mademlis et al, 2016), pictorial storyline summarization (Wang et al, 2012), timeline summarization (Wang et al, 2016b) and social multimedia summarization (Del Fabro et al, 2012;Bian et al, 2013;Schinas et al, 2015;Bian et al, 2015;Shah et al, 2015Shah et al, , 2016. When summarizing meeting recordings, sport videos and movies, such videos consist of synchronized voice, visual and captions.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multi-modal Summarization for Asynchronous Collection of Text, Image, Audio and Video

Li¹,

Zhu²,

Ma³

et al. 2017

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

The rapid increase in multimedia data transmission over the Internet necessitates the multi-modal summarization (MMS) from collections of text, image, audio and video. In this work, we propose an extractive multi-modal summarization method that can automatically generate a textual summary given a set of documents, images, audios and videos related to a specific topic. The key idea is to bridge the semantic gaps between multi-modal content. For audio information, we design an approach to selectively use its transcription. For visual information, we learn the joint representations of text and images using a neural network. Finally, all of the multimodal aspects are considered to generate the textual summary by maximizing the salience, non-redundancy, readability and coverage through the budgeted optimization of submodular functions. We further introduce an MMS corpus in English and Chinese, which is released to the public 1 . The experimental results obtained on this dataset demonstrate that our method outperforms other competitive baseline methods.

show abstract

Section: Multi-modal Summarizationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Multi-modal Summarization for Asynchronous Collection of Text, Image, Audio and Video

Li¹,

Zhu²,

Ma³

et al. 2017

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

show abstract

“…Most current highlight extraction methods utilizing game audio tend to focus on simplistic features such as audio energy or short-time zero crossing [5,6] to estimate the excitement level. Past literature on emotions and stress suggests that a number of speech production parameters can be affected by varying speech modalities [18][19][20][21].…”

Section: Excitement Measurement In Speechmentioning

confidence: 99%

“…For each frame t, the audio energy 6 (t) is extracted and later filtered using Equation 8 to obtain G 6 (t). Finally, the averaged value of G 6 (t) in the kth segment is used to compute the segmental audio energy features x i (k).…”

Section: Audio Energy Measurementioning

confidence: 99%

“…Past studies [5,6] typically consider simplistic features such as energy, zero crossing rate, and others to estimate excitement from the audio modality. Inspired by studies on emotion assessment [18][19][20], in this study, we analyze the effects of excitement on the parameters of a linear model of speech production derived from commentators' speech.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multi-modal highlight generation for sports videos using an information-theoretic excitability measure

Hasan

Bořil

Sangwan

et al. 2013

EURASIP J. Adv. Signal Process.

View full text Add to dashboard Cite

The ability to detect and organize 'hot spots' representing areas of excitement within video streams is a challenging research problem when techniques rely exclusively on video content. A generic method for sports video highlight selection is presented in this study which leverages both video/image structure as well as audio/speech properties. Processing begins where the video is partitioned into small segments and several multi-modal features are extracted from each segment. Excitability is computed based on the likelihood of the segmental features residing in certain regions of their joint probability density function space which are considered both exciting and rare. The proposed measure is used to rank order the partitioned segments to compress the overall video sequence and produce a contiguous set of highlights. Experiments are performed on baseball videos based on signal processing advancements for excitement assessment in the commentators' speech, audio energy, slow motion replay, scene cut density, and motion activity as features. Detailed analysis on correlation between user excitability and various speech production parameters is conducted and an effective scheme is designed to estimate the excitement level of commentator's speech from the sports videos. Subjective evaluation of excitability and ranking of video segments demonstrate a higher correlation with the proposed measure compared to well-established techniques indicating the effectiveness of the overall approach.

show abstract

Text-Image-Video Summary Generation Using Joint Integer Linear Programming

Jangra

Jatowt

Hasanuzzaman

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Automatically generating a summary for asynchronous data can help users to keep up with the rapid growth of multi-modal information on the Internet. However, the current multi-modal systems usually generate summaries composed of text and images. In this paper, we propose a novel research problem of text-image-video summary generation (TIVS). We first develop a multi-modal dataset containing text documents, images and videos. We then propose a novel joint integer linear programming multi-modal summarization (JILP-MMS) framework. We report the performance of our model on the developed dataset.

show abstract

Multi-modal summarization of key events and top players in sports tournament videos

Cited by 23 publications

References 7 publications

Multi-modal Summarization for Asynchronous Collection of Text, Image, Audio and Video

Multi-modal Summarization for Asynchronous Collection of Text, Image, Audio and Video

Multi-modal highlight generation for sports videos using an information-theoretic excitability measure

Text-Image-Video Summary Generation Using Joint Integer Linear Programming

Contact Info

Product

Resources

About