Histogram-Based Description of Local Space-Time Appearance

Abstract

Abstract. We introduce a novel local spatio-temporal descriptor intended to model the spatio-temporal behavior of a tracked object of interest in a general manner. The basic idea of the descriptor is the accumulation of histograms of an image function value through time. The histograms are calculated over a regular grid of patches inside the bounding box of the object and normalized to represent empirical probability distributions. The number of grid patches is fixed, so the descriptor is invariant to changes in spatial scale. Depending on the temporal complexity/details at hand, we introduce "first order STA descriptors" that describe the average distribution of a chosen image function over time, and "second order STA descriptors" that model the distribution of each histogram bin over time. We discuss entropy and χ 2 as well-suited similarity and saliency measures for our descriptors. Our experimental validation ranges from the patch-to the object-level. Our results show that STA, this simple, yet powerful novel description of local space-time appearance is well-suited to machine learning and will be useful in videoanalysis, including potential applications of object detection, tracking, and background modeling.

A review of feature selection methods with applications

“…Examples of features include histograms of oriented gradients, edge orientation histograms, Haar wavelets, raw pixels, gradient values, edges, color channels, etc. [44].…”

Patch-Level Spatial Layout for Classification and Weakly Supervised Localization

“…We evaluate the proposed approach on a public traffic sign dataset [4]. The dataset contains 3296 images acquired from the driver's perspective along local countryside roads.…”

Spatial information in classification of activity videos

“…In video activity recognition literature spatial information is often captured by various local space-time features as defined in [2], [3], [4], [5], [6], [7], [8], [9], [10], [11] and [12]. These local space-time features capture frame-wise spatial information by first detecting interest points with either interest point detectors (Harris detector, Hessian detectors, edge detector, corner detectors) or various sampling methods (dense sampling [13] or motion adaptive sampling [14]) for each frame, then spatio-temporal regions are defined around all the detected points in each frame and finally the spatio-temporal regions are described using one of the local space-time features.…”