Statistical Analysis of Visual Attentional Patterns for Video Surveillance

Abstract

Abstract. We show that the way people observe video sequences, other than what they observe, is important for the understanding and the prediction of human activities. In this study, we consider 36 surveillance videos, organized in four categories (confront, nothing, fight, play): the videos are observed by 19 people, ten of them are experienced operators and the other nine are novices, and the gaze trajectories of both populations are recorded by an eye tracking device. Due to the proved superior ability of experienced operators in predicting violence in surveillance footage, our aim is to distinguish the two classes of people, highlighting in which respect expert operators differ from novices. Extracting spatio-temporal features from the eye tracking data, and training standard machine learning classifiers, we are able to discriminate the two groups of subjects with an average accuracy of 80.26%. The idea is that expert operators are more focused on few regions of the scene, sampling them with high frequency and low predictability. This can be thought as a first step toward the advanced automated analysis of video surveillance footage, where machines imitate as best as possible the attentive mechanisms of humans.

Patterns of Saliency and Semantic Features Distinguish Gaze of Expert and Novice Viewers of Surveillance Footage

“…For example, Howard et al (2013) found that when monitoring a single scene to detect potentially suspicious events, trained CCTV operators showed greater consistency in fixation location by "knowing what to look for" compared to novices. Using the same dataset as the current study, Roffo et al (2013) found that expert operators are more likely to focus on a small number of interesting regions, sampling them with high frequency. A neuroimaging study, using…”

“…For example, Howard et al (2013) found that when monitoring a single scene to detect potentially suspicious events, trained CCTV operators showed greater consistency in fixation location by “knowing what to look for” compared to novices. Using the same dataset as the current study, Roffo et al (2013) found that expert operators are more likely to focus on a small number of interesting regions, sampling them with high frequency. A neuroimaging study, using CCTV video stimuli that have some overlap with the current stimuli, also provided converging evidence by demonstrating that a different sample of CCTV operators show increased synchronization of neural responses in certain regions of the brain (i.e., bilateral anterior superior temporal gyrus, left middle temporal gyrus, left ventral striatum, and left inferior parietal lobule) than do novices (Petrini et al, 2014).…”

“…Recent studies have shown significant differences between central and peripheral vision in the analysis of human body movements (Thurman & Lu, 2013, 2014), showing that configural cues based on the spatial arrangement of joint trajectories dominate visual processing in central vision, whereas local motion and orientation cues interact with spatial cues to influence action perception in the periphery. In addition, studies investigating surveillance videos have provided evidence that experienced CCTV operators, relative to novices, produce different goal-directed eye movement patterns when viewing surveillance video, and show greater consistency in eye-movement tracking patterns (Howard, Troscianko, Gilchrist, Behera, & Hogg, 2013; Roffo, Cristani, Pollick, Segalin, & Vittorrio, 2013). Although these studies have analyzed eye-movement characteristics associated with expertise and behavior, it remains unknown what stimulus content in surveillance videos drives the active selection of gaze shifts when viewing people’s activities with the goal of identifying intentions and whether CCTV operators develop different information-seeking strategies than novices.…”