Spatio-temporal attention model for video content analysis

Abstract

Abstract-This paper presents a new model of human attention that allows salient areas to be extracted from video frames. As automatic understanding of video semantic content is still far from being achieved, attention model tends to mimic the focus of the human visual system. Most existing approaches extract the saliency of images in order to be used in multiple applications but they are not compared to human perception.The model described here is achieved by the fusion of a static model inspired by the human system and a model of moving object detection. The static model is divided into two steps: a "retinal" filtering followed by a "cortical" decomposition. The moving object detection is carried out by a compensation of camera motion. Then we compare the attention model output for different videos with human judgment. A psychophysical experiment is proposed to compare the model with visual human perception and to validate it. The experimental results indicate that the model achieves about 88% of precision. This shows the usefulness of the scheme and its potential in future applications.

Visual-aural attention modeling for talk show video highlight detection

“…You et al inherited the ideology in [4] to propose a human perception analysis framework for video understanding based on multiple visual cues [5]. In [6], [7] the authors constructed the visual attention models and applied their models to the application of user focus detection in video frames. The attention modeling based video content analysis work is proved more consistent with human understanding and has less computational complexity.…”

“…Although they have the above advantages compared with traditional work, the current techniques on attention modeling based highlight extraction are mainly focused on the analysis of visual aspect but neglect the aural modality [5][6] [7], which is another important intrinsic information source of video. Besides, the highlights are usually simply determined as the local maximums of the linear fused attention curve [4], which didn't consider the highlight asynchronous attention influence factors such as applaud and cheer in their work.…”

“…All the abovementioned related work on human attention modeling either ignores the aural dimension or models this dimension comparatively simple [4][5][6] [7]. The audio-track usually contains immense amounts of useful information and it normally has closer link to semantic event than the visual information.…”

Spatial-temporal attention analysis for home video

“…To find regions of interest needs other methods such as those proposed in [5][6][7][8]. Zhai and Shah [5] and Guironnet et al [6] utilized static and motion information as spatial and temporal factors to obtain the attended areas. Liu and Gleicher [7] also analyzed image and motion saliency and applied it for retargeting video to small screens.…”

Low-quality image enhancement using visual attention

“…7 Itti and Koch, 8 then defined a visual attention system based on saliency maps to predict visually salient features of a scene. Chauvin et al 9 and Guironnet et al 10 proposed models inspired by the retina and the primary visual cortex cell functionalities. Corchs and Deco 11 implemented a neurodynamical model for visual attention, based on evidence from functional, neurophysiological, and psychological findings.…”