Student Abnormal Behavior Recognition in Classroom Video Based on Deep Learning

Liu, Huayong; Ao, Weidong; Hong, Jinlin

doi:10.1145/3501409.3501529

Cited by 12 publications

(6 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Students in the front or back rows have different proportions of pixels in the image [64]. Back-row-students have smaller pixels and have more serious occlusion problems [65]. As a result, the pixel scales of classroom learning behaviors are inconsistent, especially the classroom learning behaviors of back-row-students have insufficient pixels.…”

Section: Methodologies a Baseline Model Yolov8nmentioning

confidence: 99%

YOLOv8n_BT: Research on Classroom Learning Behavior Recognition Algorithm Based on Improved YOLOv8n

Liu,

Jiang,

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Classroom learning behavior recognition can provide effective technical support for teaching and learning. However, in natural classroom teaching scenarios, classroom learning behaviors are often missed or falsely detected due to character occlusion and the small object. To tackle the above issues, this study proposed an improved classroom learning behavior recognition algorithm (YOLOv8n_BT) based on YOLOv8n. On the one hand, for the occlusion problem of classroom learning behaviors, this study incorporated the BRA into the Backbone to better capture feature information; on the other hand, for the small object problem of classroom learning behaviors for back-row-students, this study expanded a Tiny Object Detection Layer (TODL) to detect small targets better. Experiments show that the BRA and the TODL can significantly improve the model performance. The YOLOv8n_BT model, which incorporated both the BRA and the TODL into the YOLOv8n(baseline) model simultaneously, has the most significant performance improvement. Compared with the YOLOv8n(baseline), the YOLOv8n_BT model improved by 3.0%, 6.7%, 5.0%, 3.6%, and 9.0% on P, R, F1, mAP50, and mAP50-90, respectively. The detection performance of YOLOv8n_BT also outperforms other state-of-the-arts.

show abstract

Section: Methodologies a Baseline Model Yolov8nmentioning

confidence: 99%

YOLOv8n_BT: Research on Classroom Learning Behavior Recognition Algorithm Based on Improved YOLOv8n

Liu,

Jiang,

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…By employing object detection to identify classroom behavior, the behavior that needs to be identified is treated directly as a target object, and the network is then utilized to extract spatial features to identify the behavior. Liu et al [24] used the YOLOv3 algorithm for student anomalous behavior recognition with the addition of RFB and SE-Res2net modules to improve the model for small target and crowd occlusion problems in the classroom environment. Tang et al [25] performed classroom behavior detection based on pictures, adding a feature pyramid structure and an attention mechanism to the YOLOv5 classroom behavior detection model to address the problem of high occlusion in the classroom environment.…”

Section: Behavior Detection In Classroom Scenariosmentioning

confidence: 99%

MSTA-SlowFast: A Student Behavior Detector for Classroom Environments

Zhang

Liang

Sun

et al. 2023

Sensors

View full text Add to dashboard Cite

Detecting students’ classroom behaviors from instructional videos is important for instructional assessment, analyzing students’ learning status, and improving teaching quality. To achieve effective detection of student classroom behavior based on videos, this paper proposes a classroom behavior detection model based on the improved SlowFast. First, a Multi-scale Spatial-Temporal Attention (MSTA) module is added to SlowFast to improve the ability of the model to extract multi-scale spatial and temporal information in the feature maps. Second, Efficient Temporal Attention (ETA) is introduced to make the model more focused on the salient features of the behavior in the temporal domain. Finally, a spatio-temporal-oriented student classroom behavior dataset is constructed. The experimental results show that, compared with SlowFast, our proposed MSTA-SlowFast has a better detection performance with mean average precision (mAP) improvement of 5.63% on the self-made classroom behavior detection dataset.

show abstract

“…The authors of [28] proposed a face tracking algorithm based on area of interest to detect students' standing behavior in the classroom, and also developed an algorithm based on skin color detection to recognize students' hand behavior. The authors of [29] introduced a cascaded RFB module in the YOLOv3 algorithm, which improves the feature extraction capability of the original network and realizes the goal of identifying small and medium-sized target students in the classroom. At the same time, the SE attention mechanism was introduced to express feature information in a finer-grained manner.…”

Section: Behavior Detection In Classroom Scenariosmentioning

confidence: 99%

Classroom Behavior Detection Based on Improved YOLOv5 Algorithm Combining Multi-Scale Feature Fusion and Attention Mechanism

et al. 2022

View full text Add to dashboard Cite

The detection of students’ behaviors in classroom can provide a guideline for assessing the effectiveness of classroom teaching. This study proposes a classroom behavior detection algorithm using an improved object detection model (i.e., YOLOv5). First, the feature pyramid structure (FPN+PAN) in the neck network of the original YOLOv5 model is combined with a weighted bidirectional feature pyramid network (BiFPN). They are subsequently processed with feature fusion of different scales of the object to mine the fine-grained features of different behaviors. Second, a spatial and channel convolutional attention mechanism (CBAM) is added between the neck network and the prediction network to make the model focus on the object information to improve the detection accuracy. Finally, the original non-maximum suppression is improved using the distance-based intersection ratio (DIoU) to improve the discrimination of occluded objects. A series of experiments were conducted on our new established dataset which includes four types of behaviors: listening, looking down, lying down, and standing. The results demonstrated that the algorithm proposed in this study can accurately detect various student behaviors, and the accuracy was higher than that of the YOLOv5 model. By comparing the effects of student behavior detection in different scenarios, the improved algorithm had an average accuracy of 89.8% and a recall of 90.4%, both of which were better than the compared detection algorithms.

show abstract

Student Abnormal Behavior Recognition in Classroom Video Based on Deep Learning

Cited by 12 publications

References 6 publications

YOLOv8n_BT: Research on Classroom Learning Behavior Recognition Algorithm Based on Improved YOLOv8n

YOLOv8n_BT: Research on Classroom Learning Behavior Recognition Algorithm Based on Improved YOLOv8n

MSTA-SlowFast: A Student Behavior Detector for Classroom Environments

Classroom Behavior Detection Based on Improved YOLOv5 Algorithm Combining Multi-Scale Feature Fusion and Attention Mechanism

Contact Info

Product

Resources

About