Improving Human Action Recognition Using Hierarchical Features And Multiple Classifier Ensembles

Abstract

This paper presents a simple, fast and efficacious system to promote the human action classification outcome using the depth action sequences. Firstly, the motion history mages (MHIs) and static history images (SHIs) are created from the front (XOY), side (YOZ) and top (XOZ) projected scenes of each depth sequence in a 3D Euclidean space through engaging the 3D Motion Trail Model (3DMTM). Then, the Local Binary Patterns (LBPs) algorithm is operated on the MHIs and SHIs to learn motion and static hierarchical features to represent the action sequence. The motion and static hierarchical feature vectors are then fed into a classifier ensemble to classify action classes, where the ensemble comprises of two classifiers. Thus, each ensemble includes a pair of Kernel-based Extreme Learning Machine (KELM) or ${\mathrm{l}}_{\mathrm{2}}$-regularized Collaborative Representation Classifier (${\mathrm{l}}_{\mathrm{2}}$-CRC) or Multi-class Support Vector Machine. To extensively assess the framework, we perform experiments on a couple of standard available datasets such as MSR-Action3D, UTD-MHAD and DHA. Experimental consequences demonstrate that the proposed approach gains a state-of-the-art recognition performance in comparison with other available approaches. Several statistical measurements on recognition results also indicate that the method achieves superiority when the hierarchical features are adopted with the KELM ensemble. In addition, to ensure real-time processing capability of the algorithm, the running time of major components is investigated. Based on machine dependency of the running time, the computational complexity of the system is also shown and compared with other methods. Experimental results and evaluation of the computational time and complexity reflect real-time compatibility and feasibility of the proposed system.

A tri‐attention enhanced graph convolutional network for skeleton‐based action recognition

“…Although the approach in Ref. [24] and our proposed approach both represent the properties of action sequences from three views, the two approaches are quite different. Specifically, the approach in Ref.…”

“…Specifically, the approach in Ref. [24] aims to obtain motion history images and static history images by calculating the pixel change of the same position in a time period, and the three views in Ref. [1] are front (XOY), side (YOZ), and top (XOZ) plane in a 3D Euclidean space (X-Y-Z) at each frame of a depth map sequence.…”

Gradient local auto-correlation features for depth human action recognition

“…Classification accuracy (%) DCSF [54] 89.3 HON4D [15] 88.9 Super Normal Vector [16] 93.1 Skeletons Lie group [17] 89.5 DMM-LBP-DF [55] 93.0 2D-CNN on DMM-Pyramid [44] 91.1 3D-CNN on DMM-Cube [44] 86.1 HOG3D + LLC [56] 90.9 Hierarchical 3D Kernel [57] 92.7 GLAC on DMM [13] 89.4 DMM-GLAC-STACOG [13] 94.8 3DHoT + MBC [58] 95.2 Subspace encoding [59] 94.06 LSTM + trust gates [60] 94.8 Extended SNV [61] 93.5 Trust Gates [62] 94.8 ST-NBNN [63] 94.8 SSTKDes [64] 95.6 3D-CNN + DHI + relief + SVM [65] 92.8 WDMM + HOG [66] 91.9 WDMM + LBP [66] 91.6 WDMM + CNN [66] 90.0 Deep activations [67] 92.3 Deep activations + attributes [67] 93.4 Hierarchical Gaussian [68] 95.6 GMHI + GSHI + CRC [69] 94.5 MHF + SHF + KELM [36] 95.97 Spatiotemporal + HMM [70] 92. Experimental evaluation of our approach on UTD-MHAD dataset is represented by Table 5.…”

“…The system is evaluated on a Desktop whose configuration includes an Intel i5-7500 Quad-core processor of 3.41 GHz frequency and a 16 GB RAM. There are six major components in the system-i.e., MHI/SHI construction, binary coded MHI generation, binary coded SHI generation, EAMF [39] 92.4 DMPP-PHOG [39] 95.0 DMM-LBP-DF [55] 91.3 Multi-temporal DMM [71] 95.44 3DHoT + MBC [58] 96.69 Hierarchical Gaussian [68] 97.96 MHF + SHF + MSVM [36] 96.09 MHF + SHF + KELM [36] 98 7. Note that the system needs less than one second (i.e., 731.43 ± 48.83 ms) to process 40 depth frames.…”

“…The motion features of an action through the simplified depth motion maps were extracted by works in DMM [14], DMM-CT-HOG [34], DLE [35]. The texture features extracted by LBP were utilized in [36]. Recently, Dhiman et al [37] introduced Zernike moments and R-transform to create a powerful feature vector for abnormal action detection.…”