A comparison of students' emotional self-reports with automated facial emotion recognition in a reading situation

Hirt, Franziska S.; Moser, Ivan; Werlen, Egon; Imhof, Christof; Bergamin, Per

doi:10.1145/3284179.3284230

Cited by 3 publications

(2 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To answer the first research question, the high degree of consistency between the emotional self-reports and the automatic system results, around 70%, in contrast with Hirt et al (2018), suggests that the software is reasonably reliable to determine the emotional valence of the students in the learning context. However, only 60% of the video material collected is recognized and processed by the emotional valence recognition software, which emphasizes the need for adequate framed facial images for the AI system to work (Grm et al, 2018).…”

Section: Discussionmentioning

confidence: 99%

“…Furthermore, learning contexts involve complexity that does not always allow for the adequate recording of participants' faces, which can limit the reliability of such systems (Hirt et al, 2018). The use of hybrid convolutional neural networks, combining facial expressions with hand gestures and body postures, can minimize this issue (Ashwin & Ram Mohana Reddy, 2020), although the correct interpretation of an emotional expression, as seen before, poses a complex process involving multiple parameters.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The first steps for adapting an artificial intelligence emotion expression recognition software for emotional management in the educational context

Herrero

Gomez-Donoso

Vila

2023

Brit J Educational Tech

View full text Add to dashboard Cite

To test the suitability of an automatic system for emotional management in the classroom following the control‐value theory of achievement emotions (CVT) framework, the performance of an emotional expression recognition software of our creation is evaluated in an online synchronous context. Sixty students from the Faculty of Education at the University of Alicante participated in 16 educational activities recording close‐ups of their faces and completing the AEQ emotional self‐report, as well as detailed reports from the subsequent review of their videos. In addition, they completed the VCQ‐36 test to measure their volitional competencies and relate their influence on their emotional response. The results indicate a high coherence between the emotional expressions detected by the automatic system and the detailed emotional self‐reports, but insufficient precision to meet the CVT requirements. On the other hand, both the AEQ test results and the emotion expression recognition software suggest students' preference for participative activities as opposed to passive ones. Meanwhile, statistical analysis results indicate that volitional competencies seem to influence the emotional response of students in the educational context, although the AI system does not show sufficient sensitivity in this field. Implications and limitations of this study for future work are discussed. Practitioner notesWhat is already known about this topic Student motivation and involvement in the learning process are highly related to appropriate emotional regulation, which can be associated with particular educational activities, strategies and methodologies. Deep learning technology based on convolutional neural networks feeds automatic systems focused on facial expression recognition from image analysis. What this paper adds There is high coherence between the emotional expressions detected by the AI system and the students' emotional self‐reports, but the AI system provides just emotional valences, insufficient to meet the CVT framework. Both emotional self‐reports and the emotion recognition software suggest students' preference for active educational activities as opposed to passive ones. Volitional competencies seem to influence the emotional response of students in the educational context. Implications for practice and/or policy It is possible to use automatic systems to effectively monitor the emotional response of students in the learning process. Only if sensitivity improved, a real‐time, easy‐to‐interpret emotional expression recognition software interface could be implemented to assist teachers with the emotional management of their classes within the CVT framework, maximizing their motivation and engagement.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%