Multi-Temporal Depth Motion Maps-Based Local Binary Patterns for 3-D Human Action Recognition

Chen, Chen; Liu, Mengyuan; Liu, Hong; Zhang, Baochang; Han, Jungong; Kehtarnavaz, Nasser

doi:10.1109/access.2017.2759058

Cited by 51 publications

(35 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Accuracy % LOP Feature [29] 42.5 STIPs(Harris3D+HOG3D) [74] 60.6 Random Occupancy Pattern [75] 64.0 Joint Position Feature [29] 68.0 STIPs (Cuboids+HOG/HOF) [76] 70.6 Local HON4D [77] 80.0 SNV [51] 86.3 DMMM [62] 81.9 DSTIP+DCSF [30] 83.6 WHDMM [66] 85.0 Actionlet Ensemble [29] 86.0 MDMMs [19] 89.0 CNN, SAE [69] 91.3 MM2DCNN [78] 71.7 MMDT [78] 82.5 Deep Poselets [79] 84.4 DMLAE [73] 67.1…”

Section: Methodsmentioning

confidence: 99%

Deep Learning of Fuzzy Weighted Multi-Resolution Depth Motion Maps with Spatial Feature Fusion for Action Recognition

et al. 2019

View full text Add to dashboard Cite

Human action recognition (HAR) is an important yet challenging task. This paper presents a novel method. First, fuzzy weight functions are used in computations of depth motion maps (DMMs). Multiple length motion information is also used. These features are referred to as fuzzy weighted multi-resolution DMMs (FWMDMMs). This formulation allows for various aspects of individual actions to be emphasized. It also helps to characterise the importance of the temporal dimension. This is important to help overcome, e.g., variations in time over which a single type of action might be performed. A deep convolutional neural network (CNN) motion model is created and trained to extract discriminative and compact features. Transfer learning is also used to extract spatial information from RGB and depth data using the AlexNet network. Different late fusion techniques are then investigated to fuse the deep motion model with the spatial network. The result is a spatial temporal HAR model. The developed approach is capable of recognising both human action and human–object interaction. Three public domain datasets are used to evaluate the proposed solution. The experimental results demonstrate the robustness of this approach compared with state-of-the art algorithms.

show abstract

Section: Methodsmentioning

confidence: 99%

Deep Learning of Fuzzy Weighted Multi-Resolution Depth Motion Maps with Spatial Feature Fusion for Action Recognition

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Based on Theorem I mentioned above, recent works have shown that the ELM model achieves good generalization performances in numerous applications, such as human action recognition [ 25 , 26 ], object tracking [ 27 ], scene classification [ 28 ], hyper-spectral imagery classification [ 29 ], etc. Inspired by these, we attempt to apply ELM for efficient and robust crack region detection.…”

Section: Background Contentmentioning

confidence: 99%

Crack Damage Detection Method via Multiple Visual Features and Efficient Multi-Task Learning Model

Wang

Zhao

Gao

et al. 2018

Sensors

View full text Add to dashboard Cite

This paper proposes an effective and efficient model for concrete crack detection. The presented work consists of two modules: multi-view image feature extraction and multi-task crack region detection. Specifically, multiple visual features (such as texture, edge, etc.) of image regions are calculated, which can suppress various background noises (such as illumination, pockmark, stripe, blurring, etc.). With the computed multiple visual features, a novel crack region detector is advocated using a multi-task learning framework, which involves restraining the variability for different crack region features and emphasizing the separability between crack region features and complex background ones. Furthermore, the extreme learning machine is utilized to construct this multi-task learning model, thereby leading to high computing efficiency and good generalization. Experimental results of the practical concrete images demonstrate that the developed algorithm can achieve favorable crack detection performance compared with traditional crack detectors.

show abstract

“…In recent years, the use of convolutional neural networks [9], [10] in visual recognition has become increasingly popular, and their excellent performance in such tasks has been demonstrated. To enable the feature extraction of human gestures, Kim et al [11] proposed a modified convolutional neural network (CNN) having a three dimensional receptive field, to generate a set of feature maps from the human gesture descriptors derived from a spatiotemporal volume.…”

Section: Introductionmentioning

confidence: 99%

Using Deep Learning in Infrared Images to Enable Human Gesture Recognition for Autonomous Vehicles

Geng

Yin

2020

IEEE Access

View full text Add to dashboard Cite

The realization of a novel human gesture recognition algorithm is essential to enable the effective collision avoidance of autonomous vehicles. Compared to visible spectrum cameras, the use of infrared imaging can enable more robust human gesture recognition in a complex environment. However, gesture recognition in infrared images has not been extensively investigated. In this work, we propose a model to detect human gestures, based on the improved YOLO-V3 network involving a saliency map as the second input channel to enhance the reuse of features and improve the network performance. Three DenseNet blocks are added before the residual components in the YOLO-V3 network to enhance the convolutional feature propagation. The saliency maps are obtained by multiscale superpixel segmentation, superpixel block clustering and cellular automata saliency detection. The obtained five scale saliency maps are fused using a Bayesian based fusion algorithm, and the final saliency image is generated. The infrared images composed of 4 main gesture classes are collected, each of which contains several approximated gestures in morphological terms. The training and testing datasets are generated, including original and augmented infrared images with a resolution of 640 × 480. The experimental results show that the proposed approach can enable real time human gesture detection for autonomous vehicles, with an average detection accuracy of 86.2%. INDEX TERMS Human gesture recognition, autonomous vehicles, deep learning approach, infrared images, saliency maps.

show abstract

Multi-Temporal Depth Motion Maps-Based Local Binary Patterns for 3-D Human Action Recognition

Cited by 51 publications

References 47 publications

Deep Learning of Fuzzy Weighted Multi-Resolution Depth Motion Maps with Spatial Feature Fusion for Action Recognition

Deep Learning of Fuzzy Weighted Multi-Resolution Depth Motion Maps with Spatial Feature Fusion for Action Recognition

Crack Damage Detection Method via Multiple Visual Features and Efficient Multi-Task Learning Model

Using Deep Learning in Infrared Images to Enable Human Gesture Recognition for Autonomous Vehicles

Contact Info

Product

Resources

About