Recognizing Students and Detecting Student Engagement with Real-Time Image Processing

Abstract

With COVID-19, formal education was interrupted in all countries and the importance of distance learning has increased. It is possible to teach any lesson with various communication tools but it is difficult to know how far this lesson reaches to the students. In this study, it is aimed to monitor the students in a classroom or in front of the computer with a camera in real time, recognizing their faces, their head poses, and scoring their distraction to detect student engagement based on their head poses and Eye Aspect Ratios. Distraction was determined by associating the students’ attention with looking at the teacher or the camera in the right direction. The success of the face recognition and head pose estimation was tested by using the UPNA Head Pose Database and, as a result of the conducted tests, the most successful result in face recognition was obtained with the Local Binary Patterns method with a 98.95% recognition rate. In the classification of student engagement as Engaged and Not Engaged, support vector machine gave results with 72.4% accuracy. The developed system will be used to recognize and monitor students in the classroom or in front of the computer, and to determine the course flow autonomously.

Bimodal Learning Engagement Recognition from Videos in the Classroom

“…Considering the complexity of an actual classroom, we introduced the power IoU loss function in YOLOv5 to detect the students and obtain a precision of 95.4% Ashwin et al [21] proposed an unobtrusive engagement recognition method using nonverbal cues that obtained 71% accuracy, and our proposed bimodal learning engagement method obtained 93.94% accuracy on the KNN classifier. Uçar et al [36] presented a model to predict students' engagement in the classroom from Kinect facial and head poses. However, the range of Kinect is small and cannot be used in a large-scale classroom.…”

Application of learning analytics for sequential patterns detection associated with moments of distraction in students in e‐learning platforms

“…This technology support mainly includes the development of learning content to instruct learning, the creation of learning environments to involve learning, the design of platforms and tools to improve learning, and the organization and standardization of learning resources to make learning content reusable and more formal. However, in a report by Affordable Colleges Online (AC Online), it is mentioned that having an educational setting through virtual technologies, students often face common distractions, such as social networks, text messages, notifications, or the presence of other people who speak or ask questions; DeCandia and Colleagues [4, 24] used a webcam to recognize students' faces, head postures, and distraction levels to assess student interest. Using the UPNA Head Pose Database, the Local Binary Patterns technique to achieve face recognition and head position estimation.…”

MultiMediate 2023: Engagement Level Detection using Audio and Video Features

“…It is also defined as "the value that a participant in an interaction attributes to the goal of being together with the other participant(s) and of continuing the interaction" [14]. Estimating the value of engagement plays a vital role across various domains including education [18], healthcare [6] and UX optimization [2].…”