Human behavior analysis in video surveillance: A Social Signal Processing perspective

Abstract

The analysis of human activities is one of the most intriguing and important open issues for the automated video surveillance community. Since few years ago, it has been handled following a mere Computer Vision and Pattern Recognition perspective, where an activity corresponded to a temporal sequence of explicit actions (run, stop, sit, walk, etc.). Even under this simplistic assumption, the issue is hard, due to the strong diversity of the people appearance, the number of individuals considered (we may monitor single individuals, groups, crowd), the variability of the environmental conditions (indoor/outdoor, different weather conditions), and the kinds of sensors employed. More recently, the automated surveillance of human activities has been faced considering a new perspective, that brings in notions and principles from the social, affective, and psychological literature, and that is called Social Signal Processing (SSP). SSP employs primarily nonverbal cues, most of them are outside of conscious awareness, like face expressions and gazing, body posture and gestures, vocal characteristics, relative distances in the space and the like. This paper is the first review analyzing this new trend, proposing a structured snapshot of the state of the art and envisaging novel challenges in the surveillance domain where the cross-pollination of Computer Science technologies and Sociology theories may offer valid investigation strategies.

Automatic behavior analysis in tag games: from traditional spaces to interactive playgrounds

“…Tracking pedestrians has benefited greatly from the modeling of social cues [16]. Chen et al learn elementary groups to infer high level context information to improve the tracking of multiple people [12].…”

BORIS: a free, versatile open‐source event‐logging software for video/audio coding and live observations

“…The use of photo and video cameras and audio recorders that can store a large amount of data is now widespread in fields like psychology, primatology, entomology, anthropology, ornithology, medicine, neuroscience and cognitive science (e.g. Leong et al 2003;Cristani et al 2013). There is also an increasing use of remote monitoring where researchers collect an impressive amount of data, which they have then to review to extract the target events (van Dam et al 2013;Li et al 2015).…”

Fuzzy human motion analysis: A review

“…Various new datasets were created exclusively for this purpose such as the IXMAS, i3DPost, MuHAVi, VideoWeb and CASIA Action. Last but not the least, [7,21,23,24,27,31,36] surveyed on the various applications of HMA such as the smart surveillance and advanced user interface for human-computer interaction. For the convenience of the readers, we summarize in Table 1 and 2 the available survey papers and their respective focuses.…”