An efficient human action recognition framework with pose-based spatiotemporal features

Agahian, Saeid; Negin, Farhood; Köse, Cemal

doi:10.1016/j.jestch.2019.04.014

Cited by 33 publications

(22 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To collect 3D skeleton data, RGB deep images are captured by the Microsoft Kinect sensor. This method is one of the most popular to estimate 3D human pose [ 5 , 16 , 18 ]. The method converts 2D image detections from multiple camera views into 3D images [ 28 , 29 , 30 ].…”

Section: Related Workmentioning

confidence: 99%

“…Most of these approaches involve the use of three-dimensional (3D) skeleton data. Agahian et al [ 5 ] introduced a framework based on 3D skeleton data for human action recognition. The main techniques in this framework include pose estimation and encoding.…”

Section: Related Workmentioning

confidence: 99%

“…Various approaches have been proposed for human action recognition using skeleton data [5,16,17,33]. Most of these approaches involve the use of three-dimensional (3D) skeleton data.…”

Section: Action Recognition Using Skeleton Datamentioning

confidence: 99%

“…In spite much research work in this domain, many challenges in action recognition have remained unresolved. Viewpoint variation, occlusion, body size variation of subjects, spatiotemporal localization of actions, interclass and intraclass variation, etc., are among those challenges [ 5 ]. In the classroom environment, student behaviors can be recorded and analyzed to evaluate teaching quality and student attitudes.…”

Section: Introductionmentioning

confidence: 99%

“…The skeleton data representation of human poses in videos is a popular technique for action recognition [ 5 , 15 , 16 , 17 , 18 , 19 ]. In this technique, the main task is to identify the skeleton data, including the detailed location of joints.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Student Behavior Recognition System for the Classroom Environment Based on Skeleton Pose Estimation and Person Detection

Lin

Ngo

Dow

et al. 2021

Sensors

View full text Add to dashboard Cite

Human action recognition has attracted considerable research attention in the field of computer vision, especially for classroom environments. However, most relevant studies have focused on one specific behavior of students. Therefore, this paper proposes a student behavior recognition system based on skeleton pose estimation and person detection. First, consecutive frames captured with a classroom camera were used as the input images of the proposed system. Then, skeleton data were collected using the OpenPose framework. An error correction scheme was proposed based on the pose estimation and person detection techniques to decrease incorrect connections in the skeleton data. The preprocessed skeleton data were subsequently used to eliminate several joints that had a weak effect on behavior classification. Second, feature extraction was performed to generate feature vectors that represent human postures. The adopted features included normalized joint locations, joint distances, and bone angles. Finally, behavior classification was conducted to recognize student behaviors. A deep neural network was constructed to classify actions, and the proposed system was able to identify the number of students in a classroom. Moreover, a system prototype was implemented to verify the feasibility of the proposed system. The experimental results indicated that the proposed scheme outperformed the skeleton-based scheme in complex situations. The proposed system had a 15.15% higher average precision and 12.15% higher average recall than the skeleton-based scheme did.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

“…Various approaches have been proposed for human action recognition using skeleton data [5,16,17,33]. Most of these approaches involve the use of three-dimensional (3D) skeleton data.…”

Section: Action Recognition Using Skeleton Datamentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Student Behavior Recognition System for the Classroom Environment Based on Skeleton Pose Estimation and Person Detection

Lin

Ngo

Dow

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

You watch once more: a more effective CNN architecture for video spatio-temporal action localization

Qin,

Chen,

Ben

et al. 2024

Multimedia Systems

View full text Add to dashboard Cite

The task of spatio-temporal action localization (STAL) needs to detect the action and position of individuals in the scene. Many works focus on how to improve the accuracy, but they usually ignore inference speed and practical applications. To address the above problems, we propose a new end-to-end spatio-temporal action localization network called You Watch Once More (YWOM). In this work, there are three measures proposed to improve the accuracy of positioning and recognition while guaranteeing the inference speed. First, a new feature fusion mechanism based on frequency channel attention (FCA) is proposed, which can effectively fuse the features extracted by different backbones. In addition, a new loss function is proposed to speed up the regression and convergence of the bounding box. Specifically, the SIOU regression loss function instead of the smooth L1 loss function is applied to help the model converge stably. Moreover, a lateral connection mechanism is designed to apply more backbones to our network structure. The experimental results demonstrate that YWOM can achieve online inference speed and has good performance of spatio-temporal action localization tasks. YWOM has superiority over other related works including YOWO on the UCF101-24 dataset. The frame-mAP and the video-mAP (0.2) are improved by 4.23% and 1.63% respectively.

show abstract

A scalable real-time computer vision system for student posture detection in smart classrooms

Huang,

Zhou

2023

Educ Inf Technol

View full text Add to dashboard Cite

An efficient human action recognition framework with pose-based spatiotemporal features

Cited by 33 publications

References 26 publications

Student Behavior Recognition System for the Classroom Environment Based on Skeleton Pose Estimation and Person Detection

Student Behavior Recognition System for the Classroom Environment Based on Skeleton Pose Estimation and Person Detection

You watch once more: a more effective CNN architecture for video spatio-temporal action localization

A scalable real-time computer vision system for student posture detection in smart classrooms

Contact Info

Product

Resources

About