3DFCNN: Real-Time Action Recognition using 3D Deep Neural Networks with Raw Depth Information

Sánchez-Caballero, A.; López-Diz, Sergio de; Fuentes-Jimenez, David; Losada, Cristina; Marrón, M.; Casillas-Pérez, David; Sarker, Mohammad Ibrahim

doi:10.48550/arxiv.2006.07743

Cited by 4 publications

(4 citation statements)

References 71 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our SequentialPoint-Net achieves state-of-the-art performance among all methods on the cross-view test setting and results in similar recognition accuracy as PSTNet on the cross-subject test setting. [56] depth 84.6 87.3 3DFCNN(2020) [57] depth 78.1 80.4 Stateful ConvLSTM(2020) [58] depth 80.4 79.9…”

Section: Comparison With the State-of-the-art Methodsmentioning

confidence: 99%

SequentialPointNet: A strong frame-level parallel point cloud sequence network for 3D action recognition

Li¹,

Huang²,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Point cloud sequences of 3D human actions exhibit unordered intra-frame spatial information and ordered interframe temporal information. In order to capture the spatiotemporal structures of the point cloud sequences, cross-frame spatio-temporal local neighborhoods around the centroids are usually constructed. However, the computationally expensive construction procedure of spatio-temporal local neighborhoods severely limits the parallelism of models. Moreover, it is unreasonable to treat spatial and temporal information equally in spatio-temporal local learning, because human actions are complicated along the spatial dimensions and simple along the temporal dimension. In this paper, to avoid spatio-temporal local encoding, we propose a strong parallelized point cloud sequence network referred to as SequentialPointNet for 3D action recognition. SequentialPointNet is composed of two serial modules, i.e., an intra-frame appearance encoding module and an inter-frame motion encoding module. For modeling the strong spatial structures of human actions, each point cloud frame is processed in parallel in the intra-frame appearance encoding module and the feature vector of each frame is output to form a feature vector sequence that characterizes static appearance changes along the temporal dimension. For modeling the weak temporal changes of human actions, in the inter-frame motion encoding module, the temporal position encoding and the hierarchical pyramid pooling strategy are implemented on the feature vector sequence. In addition, in order to better explore spatio-temporal content, multiple level features of human movements are aggregated before performing the end-to-end 3D action recognition. Extensive experiments conducted on three public datasets show that SequentialPointNet outperforms stateof-the-art approaches. SequentialPointNet's code is available at https://github.com/XingLi1012/SequentialPointNet.git.

show abstract

Section: Comparison With the State-of-the-art Methodsmentioning

confidence: 99%

SequentialPointNet: A strong frame-level parallel point cloud sequence network for 3D action recognition

Li¹,

Huang²,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In [270], multi-view dynamic images were extracted through multi-view projection in depth videos for action characterization. An end-to-end 3D Fully CNN (3D-FCNN) was introduced in [271], which can automatically encode spatio-temporal information without pre-processing. In their subsequent work [272], a variant of LSTM, named stateful ConvLSTM, was further introduced to address the problem of memory limitation during video processing, which can be used to perform HAR from long and complex videos effectively.…”

Section: Deep Learning Methodsmentioning

confidence: 99%

Human Action Recognition from Various Data Modalities: A Review

Sun,

Ke,

Rahmani

et al. 2020

Preprint

View full text Add to dashboard Cite

Human Action Recognition (HAR), aiming to understand human behaviors and then assign category labels, has a wide range of applications, and thus has been attracting increasing attention in the field of computer vision. Generally, human actions can be represented using various data modalities, such as RGB, skeleton, depth, infrared sequence, point cloud, event stream, audio, acceleration, radar, and WiFi, etc., which encode different sources of useful yet distinct information and have various advantages and application scenarios. Consequently, lots of existing works have attempted to investigate different types of approaches for HAR using various modalities. In this paper, we give a comprehensive survey for HAR from the perspective of the input data modalities. Specifically, we review both the hand-crafted feature-based and deep learning-based methods for single data modalities, and also review the methods based on multiple modalities, including the fusion-based frameworks and the co-learning-based approaches. The current benchmark datasets for HAR are also introduced. Finally, we discuss some potentially important research directions in this area.

show abstract

“…In [57], multi-view dynamic images were extracted through multiview projections from depth videos for action recognition. In order to effectively capture spatio-temporal information in depth videos, Sanchez et al [269] proposed a 3D fully CNN architecture for HAR. In their subsequent work [270], a variant of LSTM unit was introduced to address the problem of memory limitation during video processing, which can be used to perform HAR from long and complex videos.…”

Section: Depth Modalitymentioning

confidence: 99%

Human Action Recognition From Various Data Modalities: A Review

Sun

Rahmani

et al. 2022

IEEE Trans. Pattern Anal. Mach. Intell.

286

View full text Add to dashboard Cite

Human Action Recognition (HAR) aims to understand human behavior and assign a label to each action. It has a wide range of applications, and therefore has been attracting increasing attention in the field of computer vision. Human actions can be represented using various data modalities, such as RGB, skeleton, depth, infrared, point cloud, event stream, audio, acceleration, radar, and WiFi signal, which encode different sources of useful yet distinct information and have various advantages depending on the application scenarios. Consequently, lots of existing works have attempted to investigate different types of approaches for HAR using various modalities. In this paper, we present a comprehensive survey of recent progress in deep learning methods for HAR based on the type of input data modality. Specifically, we review the current mainstream deep learning methods for single data modalities and multiple data modalities, including the fusion-based and the co-learning-based frameworks. We also present comparative results on several benchmark datasets for HAR, together with insightful observations and inspiring future research directions.

show abstract

3DFCNN: Real-Time Action Recognition using 3D Deep Neural Networks with Raw Depth Information

Cited by 4 publications

References 71 publications

SequentialPointNet: A strong frame-level parallel point cloud sequence network for 3D action recognition

SequentialPointNet: A strong frame-level parallel point cloud sequence network for 3D action recognition

Human Action Recognition from Various Data Modalities: A Review

Human Action Recognition From Various Data Modalities: A Review

Contact Info

Product

Resources

About