Exploiting stereoscopic disparity for augmenting human activity recognition performance

Abstract

General rightsThis document is made available in accordance with publisher policies. Please cite only the published version using the reference above. Full terms of use are available: http://www.bristol.ac.uk/pure/about/ebr-terms Abstract-This work investigates several ways to exploit scene depth information, implicitly available through the modality of stereoscopic disparity in 3D videos, with the purpose of augmenting performance in the problem of recognizing complex human activities in natural settings. The standard state-of-the-art activity recognition algorithmic pipeline consists in the consecutive stages of video description, video representation and video classification. Multimodal, depth-aware modifications to standard methods are being proposed and studied, both for video description and for video representation, that indirectly incorporate scene geometry information derived from stereo disparity. At the description level, this is made possible by suitably manipulating video interest points based on disparity data. At the representation level, the followed approach represents each video by multiple vectors corresponding to different disparity zones, resulting in multiple activity descriptions defined by disparity characteristics. In both cases, a scene segmentation is thus implicitly implemented, based on the distance of each imaged object from the camera during video acquisition. The investigated approaches are flexible and able to cooperate with any monocular low-level feature descriptor. They are evaluated using a publicly available activity recognition dataset of unconstrained stereoscopic 3D videos, consisting in extracts from Hollywood movies, and compared both against competing depth-aware approaches and a state-of-the-art monocular algorithm. Quantitative evaluation reveals that some of the examined approaches achieve state-of-the-art performance.

High-Level Multiple-UAV Cinematography Tools for Covering Outdoor Events

“…For example, unexpected/unplanned events might occur during live coverage (or even expected events at unknown time-instances), including e.g., leader break-away, crashes, falls, accidents. A number of these events may be automatically detected visually, by on-the-fly activity recognition [37], [38], [39], [40], [41], [42], or activity video summarization [43], [44], [45], [46], [47], [48], [49] systems. Alternatively, they can be manually annotated by a director that overviews the coverage.…”

Disparity-Augmented Trajectories for Human Activity Recognition

“…Mademlis et al [23] used disparity information to calculate HOG, HOF and MBH in different disparity zones before encoding them for activity recognition. Although their method improves the performance, but their use of disparity is limited to few disparity zones.…”

Summarization of human activity videos via low-rank approximation

“…Each Di, composed of Pi 128K-dimensional description vectors, is then transformed into a single Kc-dimensional BoF visual word histogram di [14], where c is a codebooks size parameter. The adoption of the BoF approach was motivated by its proven suitability for the representation of human activities, since it discards most of the spatial information and thus provides partial invariance to changes in camera viewpoint, number of human subjects, scale, rotation and occluded object parts [15].…”