Human Distraction Detection from Video Stream Using Artificial Emotional Intelligence

Abstract

This paper addresses the problem of identifying certain human behavior such as distraction and also predicting the pattern of it. This paper proposes an artificial emotional intelligent or emotional AI algorithm to detect any change in visual attention for individuals. Simply, this algorithm detects human's attentive and distracted periods from video stream. The algorithm uses deviation of normal facial alignment to identify any change in attentive and distractive activities, e.g., looking to a different direction, speaking, yawning, sleeping, attention deficit hyperactivity and so on. For detecting facial deviation we use facial landmarks but, not all landmarks are related to any change in human behavior. This paper proposes an attribute model to identify relevant attributes that best defines human's distraction using necessary facial landmark deviations. Once the change in those attributes is identified, the deviations are evaluated against a threshold based emotional AI model in order to detect any change in the corresponding behavior. These changes are then evaluated using time constraints to detect attention levels. Finally, another threshold model against the attention level is used to recognize inattentiveness. Our proposed algorithm is evaluated using video recording of human classroom learning activity to identify inattentive learners. Experimental results show that this algorithm can successfully identify the change in human attention which can be used as a learner or driver distraction detector. It can also be very useful for human distraction detection, adaptive learning and human computer interaction. This algorithm can also be used for early attention deficit hyperactivity disorder (ADHD) or dyslexia detection among patients.

Student attentiveness analysis in virtual classroom using distraction, drowsiness and emotion detection

“…The resulting points are then used to calculate the inclination in the x and y coordinates, determining whether the learner's head is deviating from the specified thresholds, indicating a lack of focus. Additionally, a study by [7] proposes distraction detection using both head pose and eye direction. These elements utilize relevant facial landmarks from the Dlib library, providing concise video analysis results.…”

“…After obtaining relevant facial landmarks from the face, selected landmark points used to detect eye and lip movements are analyzed for one-time computation. According to [5] and [7], the Eye Aspect Ratio (EAR) and Yawn Aspect Ratio (YAR) help determine whether a learner is feeling sleepy, representing a sign of drowsiness. However, the study by [5] only extracts frames where a person is yawning or their eyes are closed when EAR and YAR cross specified thresholds.…”

“…However, the study by [5] only extracts frames where a person is yawning or their eyes are closed when EAR and YAR cross specified thresholds. Meanwhile, the study by [7] can also monitor the duration of drowsiness over time by statistically defining the threshold time period. As states of eyes being closed cannot reliably recognize drowsiness in the early stages, a study by [6] employs a method using the Hierarchical Multiscale Long Short-Term Memory (HM-LSTM) to detect drowsiness signs at all levels, achieving higher accuracy than human judgment.…”

“…4b), including: (1) the left corner of the left eye, (2) the right corner of the right eye, (3) the tip of the nose, (4) the left corner of the mouth, (5) the right corner of the mouth, and (6) the chin. Our computation of the three Euler angles representing head orientation followed the methodology outlined in [7], employing Equations 1 and 2 to determine the camera coordinates (Cx, Cy, Cz) relative to the selected six facial landmarks, and subsequently deriving the rotation and translation vectors.…”