Delving Deep Into One-Shot Skeleton-Based Action Recognition With Diverse Occlusions

Peng, Kunyu; Roitberg, Alina; Yang, Kailun; Zhang, Jiaming; Stiefelhagen, Rainer

doi:10.1109/tmm.2023.3235300

Cited by 20 publications

(4 citation statements)

References 91 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Te results are compared with those of similar approaches used in recent years as shown in Table 3. Te method can also compete with deep learningbased methods that have emerged in recent years for action recognition [101][102][103].…”

Section: Comparisonmentioning

confidence: 99%

Channel Attention-Based Approach with Autoencoder Network for Human Action Recognition in Low-Resolution Frames

Dastbaravardeh,

Askarpour,

Saberi Anari

et al. 2024

International Journal of Intelligent Systems

View full text Add to dashboard Cite

Action recognition (AR) has many applications, including surveillance, health/disabilities care, man-machine interactions, video-content-based monitoring, and activity recognition. Because human action videos contain a large number of frames, implemented models must minimize computation by reducing the number, size, and resolution of frames. We propose an improved method for detecting human actions in low-size and low-resolution videos by employing convolutional neural networks (CNNs) with channel attention mechanisms (CAMs) and autoencoders (AEs). By enhancing blocks with more representative features, convolutional layers extract discriminating features from various networks. Additionally, we use random sampling of frames before main processing to improve accuracy while employing less data. The goal is to increase performance while overcoming challenges such as overfitting, computational complexity, and uncertainty by utilizing CNN-CAM and AE. Identifying patterns and features associated with selective high-level performance is the next step. To validate the method, low-resolution and low-size video frames were used in the UCF50, UCF101, and HMDB51 datasets. Additionally, the algorithm has relatively minimal computational complexity. Consequently, the proposed method performs satisfactorily compared to other similar methods. It has accuracy estimates of 77.29, 98.87, and 97.16%, respectively, for HMDB51, UCF50, and UCF101 datasets. These results indicate that the method can effectively classify human actions. Furthermore, the proposed method can be used as a processing model for low-resolution and low-size video frames.

show abstract

Section: Comparisonmentioning

confidence: 99%

Channel Attention-Based Approach with Autoencoder Network for Human Action Recognition in Low-Resolution Frames

Dastbaravardeh,

Askarpour,

Saberi Anari

et al. 2024

International Journal of Intelligent Systems

View full text Add to dashboard Cite

show abstract

“…Additionally, it incorporates a graph neural network (GNN) to improve performance by considering the relationships between samples. Peng et al [28] discuss skeleton-based one-shot action recognition (SOAR), which explicitly addresses occlusions. They generate diverse forms of occlusions using realistic furniture models.…”

Section: Technological Trends In the Video Action Classification Modelmentioning

confidence: 99%

Deep Learning-Based Action Classification Using One-Shot Object Detection

Yoo,

Lee,

Chung

2023

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

Deep learning-based action classification technology has been applied to various fields, such as social safety, medical services, and sports. Analyzing an action on a practical level requires tracking multiple human bodies in an image in real-time and simultaneously classifying their actions. There are various related studies on the real-time classification of actions in an image. However, existing deep learning-based action classification models have prolonged response speeds, so there is a limit to real-time analysis. In addition, it has low accuracy of action of each object if multiple objects appear in the image. Also, it needs to be improved since it has a memory overhead in processing image data. Deep learning-based action classification using one-shot object detection is proposed to overcome the limitations of multiframe-based analysis technology. The proposed method uses a one-shot object detection model and a multi-object tracking algorithm to detect and track multiple objects in the image. Then, a deep learning-based pattern classification model is used to classify the body action of the object in the image by reducing the data for each object to an action vector. Compared to the existing studies, the constructed model shows higher accuracy of 74.95%, and in terms of speed, it offered better performance than the current studies at 0.234 s per frame. The proposed model makes it possible to classify some actions only through action vector learning without additional image learning because of the vector learning feature of the posterior neural network. Therefore, it is expected to contribute significantly to commercializing realistic streaming data analysis technologies, such as CCTV.

show abstract

“…Many works focus on the classification setting, such as for images ([4,13,19,20] or for labeling activities in videos [21,22]). In Deep CORAL [13], the authors aim to improve the model performance on the OOD data based on access to the ID data and some OOD data.…”

Section: Domain Adaptationmentioning

confidence: 99%

“…A similar idea of aligning certain views of the samples from the source and target domain, but for videos, is used in [22]. Other works try to address domain adaptation for classification in a way that relies on having access to categorical labels [19][20][21]. Those techniques are not applicable for next-frame prediction, however.…”

Section: Domain Adaptationmentioning

confidence: 99%

Causal Factor Disentanglement for Few-Shot Domain Adaptation in Video Prediction

Cornille,

Laenen,

Sun

et al. 2023

Entropy

View full text Add to dashboard Cite

An important challenge in machine learning is performing with accuracy when few training samples are available from the target distribution. If a large number of training samples from a related distribution are available, transfer learning can be used to improve the performance. This paper investigates how to do transfer learning more effectively if the source and target distributions are related through a Sparse Mechanism Shift for the application of next-frame prediction. We create Sparse Mechanism Shift-TempoRal Intervened Sequences (SMS-TRIS), a benchmark to evaluate transfer learning for next-frame prediction derived from the TRIS datasets. We then propose to exploit the Sparse Mechanism Shift property of the distribution shift by disentangling the model parameters with regard to the true causal mechanisms underlying the data. We use the Causal Identifiability from TempoRal Intervened Sequences (CITRIS) model to achieve this disentanglement via causal representation learning. We show that encouraging disentanglement with the CITRIS extensions can improve performance, but their effectiveness varies depending on the dataset and backbone used. We find that it is effective only when encouraging disentanglement actually succeeds in increasing disentanglement. We also show that an alternative method designed for domain adaptation does not help, indicating the challenging nature of the SMS-TRIS benchmark.

show abstract

Delving Deep Into One-Shot Skeleton-Based Action Recognition With Diverse Occlusions

Cited by 20 publications

References 91 publications

Channel Attention-Based Approach with Autoencoder Network for Human Action Recognition in Low-Resolution Frames

Channel Attention-Based Approach with Autoencoder Network for Human Action Recognition in Low-Resolution Frames

Deep Learning-Based Action Classification Using One-Shot Object Detection

Causal Factor Disentanglement for Few-Shot Domain Adaptation in Video Prediction

Contact Info

Product

Resources

About