Three‐dimensional human activity recognition by forming a movement polygon using posture skeletal data from depth sensor

Abstract

Human activity recognition in real time is a challenging task. Recently, a plethora of studies has been proposed using deep learning architectures.The implementation of these architectures requires the high computing power of the machine and a massive database. However, handcrafted features-based machine learning models need less computing power and very accurate where features are effectively extracted. In this study, we propose a handcrafted model based on three-dimensional sequential skeleton data. The human body skeleton movement over a frame is computed through joint positions in a frame. The joints of these skeletal frames are projected into two-dimensional space, forming a "movement polygon."These polygons are further transformed into a one-dimensional space by computing amplitudes at different angles from the centroid of polygons.The feature vector is formed by the sampling of these amplitudes at different angles. The performance of the algorithm is evaluated using a support vector machine on four public datasets: MSR Action3D, Berkeley MHAD, TST Fall Detection, and NTU-RGB+D, and the highest accuracies achieved on these datasets are 94.13%, 93.34%, 95.7%, and 86.8%, respectively. These accuracies are compared with similar state-of-the-art and show superior performance.

Motion-Driven Spatial and Temporal Adaptive High-Resolution Graph Convolutional Networks for Skeleton-Based Action Recognition

“…Year Accuracy(%) ST-GCN [25] 2018 83.27 SPMFs [78] 2018 98.05 2s-AGCN [26] 2019 88.36 MeteorNet [79] 2019 88.5 UnifiedDeep [80] 2019 97.98 Movement polygon [13] 2020 94.13 P4Transformer [55] 2021 90.94 PSTNet [81] 2021 91.2 MMDNN [82] 2021 91.94 RIAC-LSTM [23] 2021 98.06 Complex Network+LSTM [83] 2022 90.7 SequentialPointNet [56] 2022 91.94 2s-MS&TA-HGCN-FC(ours) 90.54 4s-MS&TA-HGCN-FC(ours) 92. 73 depth-based methods (e.g., MMDNN [82]).…”

“…Early skeleton-based methods focus on extracting handcrafted features [11]- [13], or encode the skeleton data into sequential vectors or pseudo images, and then model and classify them with recurrent neural networks (RNN) [14], [15], long short-term memory (LSTM) networks [16]- [18], or convolutional neural networks (CNN) [19]- [24]. However, these methods either break the natural spatial graph structure of skeleton data or make it difficult to extract temporal features, and thus cannot fully model the complex spatiotemporal configurations and correlations of the body joints for HAR [1].…”

A skeleton-based method for exercise recognition based on 3D coordinates of human joints

“…Since skeleton sequences only include pose information, they are immune to external contextual influences such as changes in lighting or background [33]. Vishwakarma and Jain [34] focused on developing a method that uses skeletal pose data from depth sensors to create a so-called "motion polygon". This technique aims to improve the accuracy and efficiency of human action recognition systems.…”

“…This technique aims to improve the accuracy and efficiency of human action recognition systems. He uses special algorithms to process skeletal pose data, which enables efficient recognition and classification of different types of movements and actions [34].…”

A Work-Related Musculoskeletal Disorders (WMSDs) Risk-Assessment System Using a Single-View Pose Estimation Model

“…An RGB-D camera represented as Microsoft’s Kinect can obtain depth information in addition to the color information of the RGB camera. In previous studies, this technology was used to capture human activity recognition and assess postures [ 14 , 15 , 16 , 35 , 36 , 37 , 38 , 39 ], and it is also used to build datasets [ 40 ].…”