A Novel Attention-Based Key-Frame Determination Method

Abstract

This paper presents a novel key-frame detection method that combines the visual saliency-based attention features with the contextual game status information for sports videos. Two critical issues of the attention-based video content analysis are addressed: 1) the visual attention characteristics when a user is watching a video clip and 2) extracting the degree of excitement about the on-going game status. First, the objectoriented visual attention map and the algorithm of determining the contextual attention are presented. The procedure of the contextual inference is used to simulate how the game status attracts the viewers. Second, a fusion methodology of visual and contextual attention analysis based on the characteristics of human excitement is introduced. In addition, the amount of key-frames is determined by using the contextual attention score, while the key-frame determination depends on integrating all the visual attention scores. In experimental results, it demonstrates the robustness of the proposed system for basketball and baseball programs.Index Terms-Adaptive key-frame rate, attention modeling, content-based analysis, contextual attention, key-frame determination, visual attention model.

Movie Video Summarization- Generating Personalized Summaries Using Spatiotemporal Salient Region Detection

“…Similartothepreviousapproaches,LaiandYi(2012)computedstaticattentionvaluesusingcolor contrastandtexturecontrastwheredynamicattentionscoresarecomputedusingmotionintensityand motionorientationconsistency.Thesetwoattentionvaluesarefusedtogenerateanattentioncurve wherethehighlysalientkeyframesareextracted. Shih(2013)usedfacialandcenterbiasattentioncues alongwiththespatialandtemporalattentionfeatures. Salehin,andPaul(2015)exploitedforeground information,motionandvisualsaliencytoextractthemovingobjectsforsummarizationofsurveillance video.TheauthorsexploitedtheGraphbasedVisualSaliencyapproach(GBVS) (Hareletal.,2007) toestimatesalientregionsinsurveillancevideos.…”

“…In addition to color contrast and color distribution cues, another widely used prior called center prior (Wangetal.,2011;Shih,2013)isalsoincorporated.Accordingtothecharacteristicsofhuman perceptualsystem,theregionclosertothecenterofthescreenwillattractthemoreattention (Shih, 2013).Sincesalientobjectsareplacedneartheimagecentermostofthetime,thecenterpriorgives moreweighttotheregionsthatarenearertheimagecenterthantheregionsneartheimageboundaries. AcenterpriorcueisgeneratedusingaGaussianfunctionbasedonapatch'sdistancefromthecenter ofvideoframe.Thecenterpriorweightforapatchiscomputedas:…”

Scalable gastroscopic video summarization via similar-inhibition dictionary selection

“…By solving a specifically designed objective function under selection criteria, the optimal data points are selected as key frames. For instance, Shih [23] combines the visual saliency-based attention features with the contextual game status information of sports videos and selects the key frames with the maximal visual attention scores. Liu et al [24] model the summarization procedure as a maximum a posterior problem by integrating both shot boundary detection and key frame selection.…”

“…For the gastroscopic video, clinicians tend to focus on the abnormal lesions, which reflect the human psychological preferences [23], i.e., attention. Therefore, we take both gaze and content change information as the attention prior knowledge to reduce the gap between low-level features and high-level concepts, aiming to make the result of summarization fit the clinicians' actual preferences better.…”

“…Gaze prior: The human eye is usually attracted by the salient regions of a significant color distribution or strong contrast in the frame [23]. The locations of these salient regions indicate whether the corresponding frames can provide sufficient contents and better observation angles for clinicians to make the final diagnosis.…”