A user attention model for video summarization

Ma, Yufei; Lu, Lie; Zhang, Hong-Jiang; Li, Mingjing

doi:10.1145/641113.641116

Cited by 156 publications

(240 citation statements)

References 0 publications

Supporting

Mentioning

237

Contrasting

Unclassified

Order By: Relevance

“…All visual attention algorithms generate a saliency map with a predicted pixel-level saliency. This saliency map is thresholded at k=1, 3,5,10,15,20,25 and 30 percent to obtain binary saliency maps. The percentage of human fixations contained within each binary map is the performance measure.…”

Section: Performancementioning

confidence: 99%

“…Therefore, understanding the manner in which humans process visual stimuli in a free viewing scenario has been an interesting problem in the scientific and engineering community. Several applications in computer vision (object recognition [1], visual tracking [2], text detection [3]), graphics (non-photo realistic rendering [4] ), multimedia (video summarization [5], video compression [6]) and robotics (robot localization [7]) can benefit from better understanding of human visual attention. A detailed overview of various saliency algorithms and its applications are presented in [8].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning top down scene context for visual attention modeling in natural images

Karthikeyan

Jagadeesh

Manjunath

2013

2013 IEEE International Conference on Image Processing

View full text Add to dashboard Cite

Top down image semantics play a major role in predicting where people look in images. Present state-of-the-art approaches to model human visual attention incorporate high level object detections signifying top down image semantics in a separate channel along with other bottom up saliency channels. However, multiple objects in a scene are competing to attract our attention and this interaction is ignored in current models. To overcome this limitation, we propose a novel object context based visual attention model which incorporates the co-occurrence of multiple objects in a scene for visual attention modeling. The proposed regression based algorithm uses several high level object detectors for faces, people, cars, text and understands how their joint presence affects visual attention. Experimental results on the MIT eye tracking dataset demonstrates that the proposed method outperforms other state-of-the-art visual attention models.

show abstract

Section: Performancementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Learning top down scene context for visual attention modeling in natural images

Karthikeyan

Jagadeesh

Manjunath

2013

2013 IEEE International Conference on Image Processing

View full text Add to dashboard Cite

show abstract

“…Going one step further toward human perception of saliency, Ma et al (2002) propose a framework for detecting the salient parts of a video based on user attention models. They use motion, face, and camera attention along with audio attention models (audio saliency and speech/music) as cues to capture salient information and identify the audio and video segments to compose the summary.…”

Section: Novelty Detection and Video Summarizationmentioning

confidence: 99%

Vision, Attention Control, and Goals Creation System

Rapantzikos

Avrithis²,

Kolias³

2010

Perception-Action Cycle

View full text Add to dashboard Cite

Biological visual attention has been long studied by experts in the field of cognitive psychology. The Holy Grail of this study is the exact modeling of the interaction between the visual sensory and the process of perception. It seems that there is an informal agreement on the four important functions of the attention process: (a) the bottom-up process, which is responsible for the saliency of the input stimuli; (b) the top-down process that bias attention toward known areas or regions of predefined characteristics; (c) the attentional selection that fuses information derived from the two previous processes and enables focus; and (d) the dynamic evolution of the attentional selection process. In the following, we will outline established computational solutions for each of the four functions. OverviewMost of our impressions and memories are based on vision. Nevertheless vision mechanisms and functionalities are still not apparent. How do we perceive shape, color, or motion and how do we automatically focus on the most informative parts of the visual input? It has been long established that primates, including human, use focused attention and fast saccades to analyze visual stimuli based on the current situation or the desired goal. Neuroscientists have proven that neural information related to shape, motion, and color is transmitted through, at least, three parallel and interconnected channels to the brain rather than a single one. Hence a second question arises related to how these channels are "linked" in order to provide useful information to the brain.The Human Visual System (HVS) creates a perceptual representation of the world that is quite different than the two dimensional depiction of the retina.K. Rapantzikos ( ) Image, Video and

show abstract

“…The methods also tried to exploit the temporal component. Ma et al [4] defined a user attention model based on a motion vector field extracted from MPEG stream. This approach was used for video skimming.…”

Section: Introductionmentioning

confidence: 99%

Spatio-temporal attention model for video content analysis

Guironnet¹,

Guyader

Pellerin³

et al. 2005

IEEE International Conference on Image Processing 2005

View full text Add to dashboard Cite

Abstract-This paper presents a new model of human attention that allows salient areas to be extracted from video frames. As automatic understanding of video semantic content is still far from being achieved, attention model tends to mimic the focus of the human visual system. Most existing approaches extract the saliency of images in order to be used in multiple applications but they are not compared to human perception.The model described here is achieved by the fusion of a static model inspired by the human system and a model of moving object detection. The static model is divided into two steps: a "retinal" filtering followed by a "cortical" decomposition. The moving object detection is carried out by a compensation of camera motion. Then we compare the attention model output for different videos with human judgment. A psychophysical experiment is proposed to compare the model with visual human perception and to validate it. The experimental results indicate that the model achieves about 88% of precision. This shows the usefulness of the scheme and its potential in future applications.

show abstract

A user attention model for video summarization

Cited by 156 publications

References 0 publications

Learning top down scene context for visual attention modeling in natural images

Learning top down scene context for visual attention modeling in natural images

Vision, Attention Control, and Goals Creation System

Spatio-temporal attention model for video content analysis

Contact Info

Product

Resources

About