Classroom behavior recognition based on improved yolov3

Zhang, Yiwen; Wu, Zhe; Chen, XianJin; Dai, LongZhi; Li, Zhiyao; Zong, Xiaolan; Liu, Tao

doi:10.1109/icaie50891.2020.00029

Cited by 12 publications

(5 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The success of deep learning is mainly attributed to its ability to utilize massive amounts of data, coupled with advances in computational power and optimization techniques. Zhang Yiwen et al [35] added an attention mechanism module to the YOLO-v3 algorithm framework, thus effectively enhancing the training effect on students' class behavior characteristics and effectively in the SICAU-Classroom dataset, the classroom behavioral characteristics are accurately identified. On the other hand, Hou C K [36] used the Mel inverse spectral coefficient property of deep recurrent neural networks (RNN) to obtain automatically detected classroom instructor behaviors.…”

Section: Behavioral Characteristics Classification Methodsmentioning

confidence: 99%

Computer vision recognition in the teaching classroom: A Review

Jiang,

2024

EAI Endorsed Trans AI Robotics

View full text Add to dashboard Cite

Artificial intelligence introduces computer vision recognition into the teaching classroom, and computer vision recognition technology lays a solid foundation for the intelligent teaching classroom. Through the classroom camera video stream to the classroom student information data collection, voice, posture, facial, physiological signal data recognition analysis processing to extract and define the characteristics of student behaviour, automatic classification behaviour and then record and display student behaviour, thus effectively help teachers to grasp the students learning state and emotions, to promote the quality of teaching has far-reaching significance.

show abstract

Section: Behavioral Characteristics Classification Methodsmentioning

confidence: 99%

Computer vision recognition in the teaching classroom: A Review

Jiang,

2024

EAI Endorsed Trans AI Robotics

View full text Add to dashboard Cite

show abstract

“…Their method could accurately and quickly identify student behaviors such as fatigue, hand-ups, bowing, turning sideways, body position, and facial expressions. Zhang et al [22] proposed an improved YOLOv3 to detect student behavior in the classroom, such as sleeping, using mobile phones, taking notes, and others. The model added an attention mechanism CBAM module to the original network.…”

Section: Literature Reviewmentioning

confidence: 99%

“…CNN-10, a 10-layer deep CNN, has been used for feature extraction to recognize student behaviors [18], while the OpenPose framework has been utilized to focus on key information [19,20]. YOLOv3 uses multiple convolutional layers to isolate important features [21], and CBAM has been incorporated into YOLOv3 for feature extraction [22]. For enhanced accuracy, a weighted BiFPN combined with FPN+PAN has been used for feature extraction [23], and a CBAM has been added to it [24].…”

Section: Literature Reviewmentioning

confidence: 99%

MultiFusedNet: A Multi-Feature Fused Network of Pretrained Vision Models via Keyframes for Student Behavior Classification

Nindam,

Na,

Lee

2023

Applied Sciences

View full text Add to dashboard Cite

This research proposes a deep learning method for classifying student behavior in classrooms that follow the professional learning community teaching approach. We collected data on five student activities: hand-raising, interacting, sitting, turning around, and writing. We used the sum of absolute differences (SAD) in the LUV color space to detect scene changes. The K-means algorithm was then applied to select keyframes using the computed SAD. Next, we extracted features using multiple pretrained deep learning models from the convolutional neural network family. The pretrained models considered were InceptionV3, ResNet50V2, VGG16, and EfficientNetB7. We leveraged feature fusion, incorporating optical flow features and data augmentation techniques, to increase the necessary spatial features of selected keyframes. Finally, we classified the students’ behavior using a deep sequence model based on the bidirectional long short-term memory network with an attention mechanism (BiLSTM-AT). The proposed method with the BiLSTM-AT model can recognize behaviors from our dataset with high accuracy, precision, recall, and F1-scores of 0.97, 0.97, and 0.97, respectively. The overall accuracy was 96.67%. This high efficiency demonstrates the potential of the proposed method for classifying student behavior in classrooms.

show abstract

“…In an analysis of classroom teaching behavior, Wen et al (Zhang et al, 2020 ) constructed a new dataset called SICAU-Classroom Teaching Behavior which includes 584 images and 31,380 annotated objects. They integrated the CBAM attention mechanism module in YOLO3, enhancing the target behavior detection rate.…”

Section: Dataset Construction For Classroom Teaching Analysismentioning

confidence: 99%

“…In an analysis of classroom teaching behavior, Wen et al (Zhang et al, 2020) classroom teaching behavior datasets were mainly constructed based on classroom recordings. This approach has limitations in terms of data and source variety, however, making it difficult to comprehensively represent the intricate micro-behaviors taking place in actual classrooms.…”

Section: Dataset Construction For Classroom Teaching Analysismentioning

confidence: 99%

Multi-view and multi-scale behavior recognition algorithm based on attention mechanism

Zhang,

Chen,

Tan

et al. 2023

Front. Neurorobot.

View full text Add to dashboard Cite

Human behavior recognition plays a crucial role in the field of smart education. It offers a nuanced understanding of teaching and learning dynamics by revealing the behaviors of both teachers and students. In this study, to address the exigencies of teaching behavior analysis in smart education, we first constructed a teaching behavior analysis dataset called EuClass. EuClass contains 13 types of teacher/student behavior categories and provides multi-view, multi-scale video data for the research and practical applications of teacher/student behavior recognition. We also provide a teaching behavior analysis network containing an attention-based network and an intra-class differential representation learning module. The attention mechanism uses a two-level attention module encompassing spatial and channel dimensions. The intra-class differential representation learning module utilized a unified loss function to reduce the distance between features. Experiments conducted on the EuClass dataset and a widely used action/gesture recognition dataset, IsoGD, demonstrate the effectiveness of our method in comparison to current state-of-the-art methods, with the recognition accuracy increased by 1–2% on average.

show abstract

Classroom behavior recognition based on improved yolov3

Cited by 12 publications

References 11 publications

Computer vision recognition in the teaching classroom: A Review

Computer vision recognition in the teaching classroom: A Review

MultiFusedNet: A Multi-Feature Fused Network of Pretrained Vision Models via Keyframes for Student Behavior Classification

Multi-view and multi-scale behavior recognition algorithm based on attention mechanism

Contact Info

Product

Resources

About