Reducing Videoconferencing Fatigue through Facial Emotion Recognition

Abstract

In the last 14 months, COVID-19 made face-to-face meetings impossible and this has led to rapid growth in videoconferencing. As highly social creatures, humans strive for direct interpersonal interaction, which means that in most of these video meetings the webcam is switched on and people are “looking each other in the eyes”. However, it is far from clear what the psychological consequences of this shift to virtual face-to-face communication are and if there are methods to alleviate “videoconferencing fatigue”. We have studied the influence of emotions of meeting participants on the perceived outcome of video meetings. Our experimental setting consisted of 35 participants collaborating in eight teams over Zoom in a one semester course on Collaborative Innovation Networks in bi-weekly video meetings, where each team presented its progress. Emotion was tracked through Zoom face video snapshots using facial emotion recognition that recognized six emotions (happy, sad, fear, anger, neutral, and surprise). Our dependent variable was a score given after each presentation by all participants except the presenter. We found that the happier the speaker is, the happier and less neutral the audience is. More importantly, we found that the presentations that triggered wide swings in “fear” and “joy” among the participants are correlated with a higher rating. Our findings provide valuable input for online video presenters on how to conduct better and less tiring meetings; this will lead to a decrease in “videoconferencing fatigue”.

“Seeing and Being Seen” or Just “Seeing” in a Smart Classroom Context When Videoconferencing: A User Experience-Based Qualitative Research on the Use of Cameras

“…Recently, diverse issues have been raised in the research arena about the use of cameras when videoconferencing, such as: (1) why students turn their cameras on or off [ 25 , 26 , 35 , 36 , 37 ]; (2) the fatigue associated with videoconferencing [ 38 , 39 , 40 , 41 , 42 , 43 , 44 ] and how to cope with this issue [ 41 , 45 , 46 , 47 ] or the development of a conceptual model about this topic [ 48 ]; (3) engagement or disengagement [ 49 , 50 , 51 ], a key element when analyzing learning [ 52 , 53 ]; (4) difficulties in maintaining attention [ 41 , 54 ]; (5) the emotions that result from using cameras in synchronous learning [ 55 ], including the stress [ 38 ] and anxiety caused by videoconferencing [ 56 , 57 , 58 ]; (6) privacy concerns [ 36 , 59 , 60 , 61 ]; (7) users’ preferences and comparisons when using F2F versus online formats [ 15 , 62 , 63 ]; or (8) guidelines and recommendations for users when videoconferencing [ 20 , 21 , 49 , 64 ], and even in a broader scope, analyzing on line learning through literature review approaches [ 65 , 66 ], or examining remote teaching in terms of its strengths, weaknesses, opportunities, and challenges [ 67 ].…”

Emotion recognition method using millimetre wave radar based on deep learning

“…It is challenging to design reliable features based on computer vision or physiological signals because of the significant difference between individual pairs and emotions. Therefore, with the deepening research on emotion and the continuous development of deep learning technology, especially the successful application of deep learning in many fields, such as computer vision, natural language processing, medical security, educational services and HCI, deep learning technology has been widely applied to the new challenges of emotion recognition [10,15]. To address the limitations of the traditional methods mentioned above, this study proposes a practical deep learning framework for emotion recognition, hereafter referred to as ER-CNN.…”

Real-Time Facial Expression Recognition and Speech Tran-scripts over an on-premise Video Conference Application

“…Audience's impressions, such as sad, happy, natural, or surprised, are very helpful information for who is responsible for the meeting or the stakeholders to make a decision based on the video conference participants' feedback. The advantages of FER in decision-making have been realized as a result of the experiment findings that had been executed by Peter Gloor et al [17]. In this experimental setting, 35 people worked in eight teams over Zoom in a one-semester course on Collaborative Innovation Networks, with bi-weekly video meetings where each team presented their progress.…”