Classroom Behavior Detection Based on Improved YOLOv5 Algorithm Combining Multi-Scale Feature Fusion and Attention Mechanism

Abstract

The detection of students’ behaviors in classroom can provide a guideline for assessing the effectiveness of classroom teaching. This study proposes a classroom behavior detection algorithm using an improved object detection model (i.e., YOLOv5). First, the feature pyramid structure (FPN+PAN) in the neck network of the original YOLOv5 model is combined with a weighted bidirectional feature pyramid network (BiFPN). They are subsequently processed with feature fusion of different scales of the object to mine the fine-grained features of different behaviors. Second, a spatial and channel convolutional attention mechanism (CBAM) is added between the neck network and the prediction network to make the model focus on the object information to improve the detection accuracy. Finally, the original non-maximum suppression is improved using the distance-based intersection ratio (DIoU) to improve the discrimination of occluded objects. A series of experiments were conducted on our new established dataset which includes four types of behaviors: listening, looking down, lying down, and standing. The results demonstrated that the algorithm proposed in this study can accurately detect various student behaviors, and the accuracy was higher than that of the YOLOv5 model. By comparing the effects of student behavior detection in different scenarios, the improved algorithm had an average accuracy of 89.8% and a recall of 90.4%, both of which were better than the compared detection algorithms.

MultiFusedNet: A Multi-Feature Fused Network of Pretrained Vision Models via Keyframes for Student Behavior Classification

“…The original YOLOv4 was compared with an improved network (PANet and Faster R-CNN), and the results showed that the enhanced network can achieve better results and is suitable for student detection and recognition tasks. Tang et al [24] presented a classroom behavior detection algorithm based on an improved YOLOv5 object detection model. They combined a weighted bidirectional feature pyramid network (BiFPN) with the feature pyramid network and the path aggregation network (FPN+PAN) structure of YOLOv5.…”

“…To address the limitations of CNN-based target detection methods, their system employs a feature pyramid structure and the CARAFE lightweight operator to process multi-scale feature maps effectively and enhance the network's detection accuracy. This system achieved a significant 6.1% improvement in detection accuracy compared with the original deformable DETR model and outperforms Faster R-CNN, SSD, and YOLOv3 [21], v5 [24], and v7 [25].…”

“…YOLOv3 uses multiple convolutional layers to isolate important features [21], and CBAM has been incorporated into YOLOv3 for feature extraction [22]. For enhanced accuracy, a weighted BiFPN combined with FPN+PAN has been used for feature extraction [23], and a CBAM has been added to it [24]. In YOLOv7, the biformer attention module and Wise-IoU have been applied for feature extraction [25].…”

Pendeteksi Jumlah Orang pada Sistem Bangunan Pintar Menggunakan Algoritma You Only Look Once

“…Perkembangan metode YOLO juga sangat cepat terhitung sejak dua tahun terakhir sudah terdapat versi yang ke-7 [15]. Selain perkembangan YOLOv7 terdapat kustomisasi YOLO yang lain seperti FMD-YOLO [16], ETL-YOLO [17], Modified YOLOv5 [18], Modified YOLOv4 [19] [20]. Selain penggunaan YOLO…”

Multi-view and multi-scale behavior recognition algorithm based on attention mechanism

“…They analyzed classroom scenario characteristics and the technical intricacies of each module (task), offering baseline comparisons with mainstream datasets. Tang et al ( 2022 ) manually constructed a classroom teaching behavior detection dataset based on real classroom videos. Their dataset includes four behavior types: listening, looking down, lying down, and standing.…”