Convolutional neural network on three orthogonal planes for dynamic texture classification

Abstract

Dynamic Textures (DTs) are sequences of images of moving scenes that exhibit certain stationarity properties in time such as smoke, vegetation and fire. The analysis of DT is important for recognition, segmentation, synthesis or retrieval for a range of applications including surveillance, medical imaging and remote sensing. Deep learning methods have shown impressive results and are now the new state of the art for a wide range of computer vision tasks including image and video recognition and segmentation. In particular, Convolutional Neural Networks (CNNs) have recently proven to be well suited for texture analysis with a design similar to a filter bank approach. In this paper, we develop a new approach to DT analysis based on a CNN method applied on three orthogonal planes x y, xt and y t . We train CNNs on spatial frames and temporal slices extracted from the DT sequences and combine their outputs to obtain a competitive DT classifier. Our results on a wide range of commonly used DT classification benchmark datasets prove the robustness of our approach. Significant improvement of the state of the art is shown on the larger datasets.

Directional Beams of Dense Trajectories for Dynamic Texture Recognition

“…50-class: It can be verified that our obtained results (99.5% and 99%) are similar to methods' using deep learning techniques, i.e. PCANet-TOP [10] and DT-CNN [9]. Although our descriptor takes dimension of 12,760 bins, larger than MBSIF-TOP's [4] with 7-scale (5,376 bins), to obtain the same accuracy (99.5%), it is clear that our method is more efficient in other DT datasets (DynTex, DynTex++) in which MBSIF-TOP [4] achieves those with different multi-scale settings.…”

“…4(c)) for encoding although we reduced the min distance parameter of the tool to minimum value for extracting more trajectories (see more detail in 4.1). In LBP-based approaches, MEWLSP [18] points out the highest rate of 98.48% in this scheme, even higher than the rate of DT-CNN [9] (98.18%) using AlexNet for learning patterns. However, it is not better than ours on UCLA dataset as well has not been tested on other challenging DynTex variants (i.e.…”

“…9-class: Our proposed framework gains the best result of 99.35% compared to all existing approaches including deep-learning-based methods, DT-CNN [9] with rates of 98.05% and 98.35% using AlexNet and GoogleNet architectures respectively, (except DNGP [5], just 0.25% higher than ours).…”

“…8-class: On the more challenging 8-class scheme, our method demonstrates outstanding performance with 99.57%, the highest evaluation in comparison with the state-of-the-art approaches while the best rate of DT-CNN [9] just achieves 99.02% by the GoogleNet framework. A geometry-based method, 3D-OTF [7], also obtains nearly same ours but it failed in testing on other DT datasets.…”

“…Fourth, various geometry-based methods have been presented using fractal analysis techniques in which fractal dimension and other fractal characteristics are taken into account for DT representation: dynamic fractal spectrum (DFS) [6], Multi-fractal spectrum (MFS) [7], wavelet-based MFS descriptor [8]. Fifth, owing to outperforming results, learning-based methods have recently attracted researchers with promising techniques coming from recent advances in deep learning: Transferred ConvNet Features (TCoF) [9], PCA convolutional network (PCANet-TOP) [10], Dynamic Texture Convolutional Neural Network (DT-CNN) [9]. Lately, dictionary-learning-based methods [11,12] have also become more popular in which local DT features are figured out by kernel sparse coding.…”