Skeleton-Based Action Recognition With Key-Segment Descriptor and Temporal Step Matrix Model

Abstract

Human action recognition based on skeleton has played a key role in various computer visionrelated applications, such as smart surveillance, human-computer interaction, and medical rehabilitation. However, due to various viewing angles, diverse body sizes, and occasional noisy data, etc., this remains a challenging task. The existing deep learning-based methods require long time to train the models and may fail to provide an interpretable descriptor to code the temporal-spatial feature of the skeleton sequence. In this paper, a key-segment descriptor and a temporal step matrix model are proposed to semantically present the temporal-spatial skeleton data. First, a skeleton normalization is developed to make the skeleton sequence robust to the absolute body size and initial body orientation. Second, the normalized skeleton data is divided into skeleton segments, which are treated as the action units, combining 3D skeleton pose and the motion. Each skeleton sequence is coded as a meaningful and characteristic key segment sequence based on the key segment dictionary formed by the segments from all the training samples. Third, the temporal structure of the key segment sequence is coded into a step matrix by the proposed temporal step matrix model, and the multiscale temporal information is stored in step matrices with various steps. Experimental results on three challenging datasets demonstrate that the proposed method outperforms all the hand-crafted methods and it is comparable to recent deep learning-based methods. INDEX TERMS Skeleton-based action recognition, view alignment, scale normalization, key-segment descriptor, temporal step matrix model.

AutoGCN-Toward Generic Human Activity Recognition With Neural Architecture Search

“…Prior research on skeleton-based action recognition demonstrated the paramount importance of data preprocessing [9], [24], [25], [26], [27], [28], [29]. In the context of this study, the input features are categorized into four distinct groups, namely: 1) position P, 2) velocity V , 3) bone B ∈ {L, β}, and 4) acceleration features A, corresponding to the fundamental kinematic properties of the human body [30].…”

Graph Laplacian-Improved Convolutional Residual Autoencoder for Unsupervised Human Action and Emotion Recognition

“…It is a representation easily given by off-the-shelf body pose detectors and potentially allows to perform HAR in real time. Most of the work follows a supervised learning framework where the set of actions should be pre-defined and annotated for training a model [3], [4], [33], [34], [35], [36]. Whereas unsupervised HAR (U-HAR) approaches are in general under-performing compared to their supervised counterparts, but i) they can provide a more robust adaptation to real-world applications as they do not need re-training when the scenario of application changes and ii) they eliminate the need for very expensive and time-consuming annotation efforts.…”

Skeleton-based Approaches based on Machine Vision: A Survey

“…For the former, based on empirical knowledge of human body, the relations among joints and limbs under a particular action is calculated with explicit equations [120], [121], [122], [123], [124], [125], [126].…”