Bags of Spacetime Energies for Dynamic Scene Recognition

Abstract

This paper presents a unified bag of visual word (BoW) framework for dynamic scene recognition. The approach builds on primitive features that uniformly capture spatial and temporal orientation structure of the imagery (e.g., video), as extracted via application of a bank of spatiotemporally oriented filters. Various feature encoding techniques are investigated to abstract the primitives to an intermediate representation that is best suited to dynamic scene representation. Further, a novel approach to adaptive pooling of the encoded features is presented that captures spatial layout of the scene even while being robust to situations where camera motion and scene dynamics are confounded. The resulting overall approach has been evaluated on two standard, publically available dynamic scene datasets. The results show that in comparison to a representative set of alternatives, the proposed approach outperforms the previous state-of-the-art in classification accuracy by 10%.

Beyond Gauss: Image-Set Matching on the Riemannian Manifold of PDFs

“…Note that no state-of-the-art results has been reported in the literature on our second test protocol. Our approach outperforms the state-of-the-art result of Feichtenhofer (Feichtenhofer et al, 2014) by more than 7%. While this may be attributed in part to the CNN features, note that our approach still outperforms GDA and CDL based on the same features.…”

D3: Recognizing dynamic scenes with deep dual descriptor based on key frames and key segments

“…For a fair comparison with previous studies on Maryland and Yupenn, we followed the leave-one-out evaluation protocol and reported classification results using SVM. Table 5 shows the comparison results, which can be divided into hand-crafted features [20,21,9,16,10] may be because of their temporal encoding based on differences between two adjacent frames, which motivated this study. In contrast, our D3 d based on key segments shows good performance.…”

“…To better represent dynamic scenes, Derpanis et al [6] introduced multi-scale orientation features using 3D Gaussian third-derivative filters. The bag of features (BoF) scheme [8] was additionally applied to represent several spatiotemporal patches in dynamic scenes [9,10]. Encouraged by the promising results of convolutional neural networks (CNNs) [11,12,13], Tran et al [14] recently proposed a convolutional three-dimensional (C3D) architecture that is a spatiotemporal version of CNN.…”

“…modeling, including tremendous hand-crafted spatial features [16] or transferred CNN features [19], outperform methods involving simultaneous modeling [6,9,10,20,21]. Nevertheless, we believe that performance can still be improved of separate modeling of dynamic scenes in two aspects as outlined below.…”

Dynamic scene classification using convolutional neural networks

“…They are encoded using a learned dictionary and then dynamically pooled. The technique currently holds the highest accuracy on the two mentioned datasets [22] (VLAD) to obtain better than state of art performance for the event detection problem [30]. Off the shelf descriptors were used to obtain high score on TRECVID-MED dataset.…”