Affective database for e-learning and classroom environments using Indian students’ faces, hand gestures and body postures

Ashwin, T. S.; Guddeti, Ram Mohana Reddy

doi:10.1016/j.future.2020.02.075

Cited by 50 publications

(12 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Studies indicated that students' on-task rates are positively correlated with learning performance (Usart et al, 2013). Since the on-task state considers both observable and internal information, many studies tried to evaluate the learning state automatically through computer vision (Ashwin and Guddeti, 2020a;Gupta et al, 2019) or physiological data (Lin and Chen, 2019;Mazher et al, 2017).…”

Section: Engagement Detection Methods In the Classroommentioning

confidence: 99%

Multimodal Fast–Slow Neural Network for learning engagement evaluation

Zhang

Hung

et al. 2023

DTA

View full text Add to dashboard Cite

PurposeStudent engagement is a key factor that connects with student achievement and retention. This paper aims to identify individuals' engagement automatically in the classroom with multimodal data for supporting educational research.Design/methodology/approachThe video and electroencephalogram data of 36 undergraduates were collected to represent observable and internal information. Since different modal data have different granularity, this study proposed the Fast–Slow Neural Network (FSNN) to detect engagement through both observable and internal information, with an asynchrony structure to preserve the sequence information of data with different granularity.FindingsExperimental results show that the proposed algorithm can recognize engagement better than the traditional data fusion methods. The results are also analyzed to figure out the reasons for the better performance of the proposed FSNN.Originality/valueThis study combined multimodal data from observable and internal aspects to improve the accuracy of engagement detection in the classroom. The proposed FSNN used the asynchronous process to deal with the problem of remaining sequential information when facing multimodal data with different granularity.

show abstract

Section: Engagement Detection Methods In the Classroommentioning

confidence: 99%

Multimodal Fast–Slow Neural Network for learning engagement evaluation

Zhang

Hung

et al. 2023

DTA

View full text Add to dashboard Cite

show abstract

“…Some related works in the literature have reported the relevant students' facial expressions, which help in determining their academic affective states. The work (Ashwin & Guddeti, 2020) reported that engaged, boredom, sleepy, frustrated, confused, and neutral are the dominant affective states that students experience in their classroom learning. The study (Bian, C., et al 2018) described enjoyment, confusion, fatigue, distraction, and neutral as students' common feelings in online learning.…”

Section: Academic Affective States Definitionmentioning

confidence: 99%

An intelligent system for monitoring students' engagement in large classroom teaching through facial expression recognition

Pabba

Kumar

2021

Expert Systems

View full text Add to dashboard Cite

Students' disengagement problem has become critical in the modern scenario due to various distractions and lack of student-teacher interactions. This problem is exacerbated with large offline classrooms, where it becomes challenging for teachers to monitor students' engagement and maintain the right-level of interactions. Traditional ways of monitoring students' engagement rely on self-reporting or using physical devices, which have limitations for offline classroom use. Student's academic affective states (e.g., moods and emotions) analysis has potential for creating intelligent classrooms, which can autonomously monitor and analyse students' engagement and behaviours in real-time. In recent literature, a few computer vision based methods have been proposed, but they either work only in the e-learning domain or have limitations in real-time processing and scalability for large offline classes. This paper presents a real-time system for student group engagement monitoring by analysing their facial expressions and recognizing academic affective states: 'boredom,' 'confuse,' 'focus,' 'frustrated,' 'yawning,' and 'sleepy,' which are pertinent in the learning environment. The methodology includes certain pre-processing steps like face detection, a convolutional neural network (CNN) based facial expression recognition model, and post-processing steps like frame-wise group engagement estimation. For training the CNN model, we created a dataset of the aforementioned facial expressions from classroom lecture videos and added related samples from three publicly available datasets, BAUM-1, DAiSEE, and YawDD, to generalize the model predictions. The trained model has achieved train and test accuracy of 78.70% and 76.90%, respectively. The proposed methodology gave promising results when compared with self-reported engagement levels by students.

show abstract

“…Student performances were studied within the classroom for 350 students with 71 percent precision on the data collection of the Gold Standard Report than Cohen Kappa on non-verbal signs [22]. To study the affective states of the student on the e-learning environment with 83 percent, 76 percent accuracy on detection and classification, based on spontaneous and posed data sets [23]. Web-based machine translation tools to research students' actions among Korean-speaking language learners (Google translate, Navel Translate) [24].…”

Section: Deep Learning-based Intelligent Classroom Experiencesmentioning

confidence: 99%

Design and Implementation of Intelligent Classroom Framework Through Light-Weight Neural Networks Based on Multimodal Sensor Data Fusion Approach

Murugesan¹,

Chettiar²

2021

RIA

View full text Add to dashboard Cite

Intelligent classrooms are becoming famous. Multimodal sensor data fusion technique, where data generated from different sensors are fused to derive at some valuable insights from the classroom settings. In this paper, the proposed framework model tries to enable intelligence in a traditional Classroom Environment by experimenting on modules such as Deep Learning based Face recognition systems, Interactive Smart Mirror Assistant, Indoor Classroom Air quality monitors. Sensor hub (Jedi One) helps to visualize and analyse streaming data in real-time. Based on the proposed framework design, experimentations are carried out. The accuracy achieved in the Face Recognition System of 71% has to be increased with 80-90% by finetuning the parameters. In future, Interactive dashboards can be activated via PowerBI or Excel worksheets. Based on the questionnaire study & responses from the participants on AI & IoT systems inside the classrooms, more than 50% responded positively to support the usage of these technologies in a Classroom Environment. The future classrooms will be (DLeIC) Deep Learning enabled IoT Classrooms to lift the educational space into a new dimension. Incorporating Deep Learning on IoT systems can be a savvy and fruitful path to collaborate with generations to come.

show abstract

Affective database for e-learning and classroom environments using Indian students’ faces, hand gestures and body postures

Cited by 50 publications

References 18 publications

Multimodal Fast–Slow Neural Network for learning engagement evaluation

Multimodal Fast–Slow Neural Network for learning engagement evaluation

An intelligent system for monitoring students' engagement in large classroom teaching through facial expression recognition

Design and Implementation of Intelligent Classroom Framework Through Light-Weight Neural Networks Based on Multimodal Sensor Data Fusion Approach

Contact Info

Product

Resources

About