An emotion analysis in learning environment based on theme-specified drawing by convolutional neural network

Abstract

Emotion in the learning process can directly influence the learner's attention, memory, and cognitive activities. Several literatures indicate that hand-drawn painting could reflect the learner's emotional status. But, such an evaluation of emotional status, manually conducted by the psychologist, is usually subjective and inefficient for clinical practice. To address the issues of subjectivity and inefficiency in the painting based emotional analysis, we conducted an exploration of a painting based emotional analysis in learning environment by using convolutional neural network model. A painting image of 100 × 100 pixels was used as input for the model. The instant emotional statue of the learner was collected by filling out a questionnaire and was reviewed by a psychologist and then used as the label for training the convolutional neural network model. With the completion of convolutional, full-connected, and classification operations, the features of the painting image were learned from the underlying pixel matrix to the high-level semantic feature mapping. Then the emotional classification of the painting image could be made to reflect the learner's emotional status. Finally, the classification result by the model was compared with the result manually conducted by a psychologist to validate the model accuracy. We conducted an experiment in a university at Hangzhou, and 2,103 learners joined in the experiment. The learner was required to first fill out a questionnaire reporting emotional status in the learning process, and then to complete a theme-specified painting. Two thousand valid paintings were received and divided into training dataset (1,600) and test dataset (400). The experimental result indicated that the model achieved the accuracy of 72.1%, which confirmed the effectiveness of the model for emotional analysis.

Table_1.DOCX

“…The main methods used in the research to analyze students' emotional states were related to facial expressions, eye movements, and biosignal data (Ninaus et al, 2019;Dehbozorgi and Kunuku, 2023;Villegas-Ch et al, 2023;Yugal et al, 2023). During online lessons, monitoring systems studied real-time attention, emotions and feelings (He et al, 2022;Dehbozorgi and Kunuku, 2023).…”

“…In educational settings, the use of artificial intelligence, particularly machine learning and deep learning, has grown increasingly popular. These technologies primarily enhance the speed of analysis and the accuracy of emotion classification (He et al, 2022;Begum et al, 2023;Villegas-Ch et al, 2023;Yugal et al, 2023). Although artificial intelligence has seen significant advancements recently, various models have also been employed for speech emotion recognition to explore the relationship between emotions and academic performance (Dehbozorgi and Kunuku, 2023).…”

“…These technologies have been instrumental in identifying the impact of emotions on learning outcomes, linking positive emotions to improved cognitive processes and engagement (He et al, 2022;Villegas-Ch et al, 2023;Yugal et al, 2023).…”

“…Technologies such as AI, machine learning, and deep learning are employed to expedite emotion analysis and enhance classification accuracy in educational settings (He et al, 2022;Dehbozorgi and Kunuku, 2023). The integration of artificial intelligence (AI) into the management of emotions within education marks a significant advancement in modern teaching methods.…”

“…Additionally, the emotion-based artificial decision-making model has been shown to enhance the performance of educational agents in virtual settings (Yang and Zhen, 2014). Another approach involves the integration of emotional agents in AI-based learning environments to improve learner motivation, self-assessment, and self-motivation by improving the socioemotional climate (Gorga and Schneider, 2009), especially affective computing (Kort et al, 2001;González-Hernández et al, 2018;Ninaus et al, 2019;Shobana and Kumar, 2021;He et al, 2022;Aly et al, 2023;Villegas-Ch et al, 2023).…”