A Large-scale RGB-D Database for Arbitrary-view Human Action Recognition

Ji, Yanli; Xu, Feixiang; Yang, Yang; Shen, Fumin; Shen, Heng Tao; Zheng, Wei‐Shi

doi:10.1145/3240508.3240675

Cited by 64 publications

(45 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The size of NTU allows training deep neural networks unlike previous datasets. Very recently, Ji et al (Ji et al 2018) collected the first large-scale dataset, UESTC, that has a 360 • coverage around the performer, although still in a lab setting.…”

Section: Related Workmentioning

confidence: 99%

“…UESTC is a recent dataset (Ji et al 2018) that systematically collects 8 equally separated viewpoints that cover 360 • around a person (see Fig. 5).…”

Section: Uestc Rgb-d Varying-view 3d Action Dataset (Uestc)mentioning

confidence: 99%

“…In the following, we employ the standard protocols for UESTC and NTU datasets, and compare our performance with other works. Tables 6 and 7 compare our results to the state-of-the-art methods reported by Ji et al (2018) on the recently released UESTC dataset, on CV-I and CV-II protocols. To augment the UESTC dataset, we use the VIBE motion estimation method.…”

Section: Comparison With the State Of The Artmentioning

confidence: 99%

See 2 more Smart Citations

Synthetic Humans for Action Recognition from Unseen Viewpoints

et al. 2021

View full text Add to dashboard Cite

Although synthetic training data has been shown to be beneficial for tasks such as human pose estimation, its use for RGB human action recognition is relatively unexplored. Our goal in this work is to answer the question whether synthetic humans can improve the performance of human action recognition, with a particular focus on generalization to unseen viewpoints. We make use of the recent advances in monocular 3D human body reconstruction from real action sequences to automatically render synthetic training videos for the action labels. We make the following contributions: (1) we investigate the extent of variations and augmentations that are beneficial to improving performance at new viewpoints. We consider changes in body shape and clothing for individuals, as well as more action relevant augmentations such as non-uniform frame sampling, and interpolating between the motion of individuals performing the same action; (2) We introduce a new data generation methodology, SURREACT, that allows training of spatio-temporal CNNs for action classification; (3) We substantially improve the state-of-the-art action recognition performance on the NTU RGB+D and UESTC standard human action multi-view benchmarks; Finally, (4) we extend the augmentation approach to in-the-wild videos from a subset of the Kinetics dataset to investigate the case when only one-shot training data is available, and demonstrate improvements in this case as well.

show abstract

Section: Related Workmentioning

confidence: 99%

“…UESTC is a recent dataset (Ji et al 2018) that systematically collects 8 equally separated viewpoints that cover 360 • around a person (see Fig. 5).…”

Section: Uestc Rgb-d Varying-view 3d Action Dataset (Uestc)mentioning

confidence: 99%

Section: Comparison With the State Of The Artmentioning

confidence: 99%

See 1 more Smart Citation

Synthetic Humans for Action Recognition from Unseen Viewpoints

et al. 2021

View full text Add to dashboard Cite

show abstract

“…In Reference [ 29 ], the authors generated synthetic multi-view video sequence from one view, and then trained a 3D ResNet-50 [ 40 ] on both synthetic and real data to classify actions. Among these methods, Varol et al [ 29 ] is the only work that provides cross-view evaluation through single-view training, resulting in

accuracy on the UESTC dataset [ 46 ], which then was increased to

when they used additional synthetic multi-view data for training.…”

Section: Related Workmentioning

confidence: 99%

VI-Net—View-Invariant Quality of Human Movement Assessment

Sardari

Paiement

Hannuna

et al. 2020

Sensors

View full text Add to dashboard Cite

We propose a view-invariant method towards the assessment of the quality of human movements which does not rely on skeleton data. Our end-to-end convolutional neural network consists of two stages, where at first a view-invariant trajectory descriptor for each body joint is generated from RGB images, and then the collection of trajectories for all joints are processed by an adapted, pre-trained 2D convolutional neural network (CNN) (e.g., VGG-19 or ResNeXt-50) to learn the relationship amongst the different body parts and deliver a score for the movement quality. We release the only publicly-available, multi-view, non-skeleton, non-mocap, rehabilitation movement dataset (QMAR), and provide results for both cross-subject and cross-view scenarios on this dataset. We show that VI-Net achieves average rank correlation of 0.66 on cross-subject and 0.65 on unseen views when trained on only two views. We also evaluate the proposed method on the single-view rehabilitation dataset KIMORE and obtain 0.66 rank correlation against a baseline of 0.62.

show abstract

“…The contextual features were detected using the CNN model. In addition, Yanliet al, 2018 [ 34 ] proposed a View-guided Skeleton-CNN (VS-CNN) model for human arbitrary view and human action recognition which carries on weakening view differences by visualizing skeleton sequences and covers a larger range of view angles. Hug et al, 2019 [ 35 ] applied an action recognition model based on the transformation of the skeleton to a spatial presentation using the conversion of the distance values of two joints to color points, and they used the DenseNet CNN model for action classification.…”

Section: Related Workmentioning

confidence: 99%

Prediction of Human Activities Based on a New Structure of Skeleton Features and Deep Learning Model

Jaouedi¹,

Perales

Buades

et al. 2020

Sensors

View full text Add to dashboard Cite

The recognition of human activities is usually considered to be a simple procedure. Problems occur in complex scenes involving high speeds. Activity prediction using Artificial Intelligence (AI) by numerical analysis has attracted the attention of several researchers. Human activities are an important challenge in various fields. There are many great applications in this area, including smart homes, assistive robotics, human–computer interactions, and improvements in protection in several areas such as security, transport, education, and medicine through the control of falling or aiding in medication consumption for elderly people. The advanced enhancement and success of deep learning techniques in various computer vision applications encourage the use of these methods in video processing. The human presentation is an important challenge in the analysis of human behavior through activity. A person in a video sequence can be described by their motion, skeleton, and/or spatial characteristics. In this paper, we present a novel approach to human activity recognition from videos using the Recurrent Neural Network (RNN) for activity classification and the Convolutional Neural Network (CNN) with a new structure of the human skeleton to carry out feature presentation. The aims of this work are to improve the human presentation through the collection of different features and the exploitation of the new RNN structure for activities. The performance of the proposed approach is evaluated by the RGB-D sensor dataset CAD-60. The experimental results show the performance of the proposed approach through the average error rate obtained (4.5%).

show abstract

A Large-scale RGB-D Database for Arbitrary-view Human Action Recognition

Cited by 64 publications

References 44 publications

Synthetic Humans for Action Recognition from Unseen Viewpoints

Synthetic Humans for Action Recognition from Unseen Viewpoints

VI-Net—View-Invariant Quality of Human Movement Assessment

Prediction of Human Activities Based on a New Structure of Skeleton Features and Deep Learning Model

Contact Info

Product

Resources

About