Video Action Recognition Collaborative Learning with Dynamics via PSO-ConvNet Transformer

Abstract

Recognizing human actions in video sequences, known as Human Action Recognition (HAR), is a challenging task in pattern recognition. While Convolutional Neural Networks (ConvNets) have shown remarkable success in image recognition, they are not always directly applicable to HAR, as temporal features are critical for accurate classification. In this paper, we propose a novel dynamic PSO-ConvNet model for learning actions in videos, building on our recent work in image recognition. Our approach leverages a framework where the weight vector of each neural network represents the position of a particle in phase space, and particles share their current weight vectors and gradient estimates of the Loss function. To extend our approach to video, we integrate ConvNets with state-of-the-art temporal methods such as Transformer and Recurrent Neural Networks. Our experimental results on the UCF-101 dataset demonstrate substantial improvements of up to 9% in accuracy, which confirms the effectiveness of our proposed method. Overall, our dynamic PSO-ConvNet model provides a promising direction for improving HAR by better capturing the spatio-temporal dynamics of human actions in videos. The code is available at https://github.com/leonlha/Video-Action-Recognition-Collaborative-Learning-with-Dynamics-via-PSO-ConvNet-Transformer.

Human action recognition with transformer based on convolutional features

“…However, our method combines CNN with transform, endowing it with higher generalization ability and robustness. [43] Two stream CNN 86.10% Kim et al [44] Two stream CNN 87.50% Proposed Method CNN + transform 87.50% On the UCF 50 dataset, our method demonstrates excellent performance with an accuracy of 83.41% as shown in table 7. The LC + Multiview pooling method proposed by Liu et al [45] achieves an accuracy of 78.60% on the UCF 50 dataset, significantly lower than our method.…”

“…[42] achieves an accuracy of 54.96% on the UCF 101 dataset, significantly lower than our method. This finding further confirms the higher classification accuracy and robustness of our approach in handling complex action recognition tasks.The two-stream CNN method suggested by Nguyen et al [43] attains an accuracy of 86.10% on the UCF 101 dataset. While this result is already relatively high, it is slightly less accurate compared to our method.…”