Analyzing Perceived Empathy Based on Reaction Time in Behavioral Mimicry

Abstract

SUMMARYThis study analyzes emotions established between people while interacting in face-to-face conversation. By focusing on empathy and antipathy, especially the process by which they are perceived by external observers, this paper aims to elucidate the tendency of their perception and from it develop a computational model that realizes the automatic inference of perceived empathy/antipathy. This paper makes two main contributions. First, an experiment demonstrates that an observer's perception of an interacting pair is affected by the time lags found in their actions and reactions in facial expressions and by whether their expressions are congruent or not. For example, a congruent but delayed reaction is unlikely to be perceived as empathy. Based on our findings, we propose a probabilistic model that relates the perceived empathy/antipathy of external observers to the actions and reactions of conversation participants. An experiment is conducted on ten conversations performed by 16 women in which the perceptions of nine external observers are gathered. The results demonstrate that timing cues are useful in improving the inference performance, especially for perceived antipathy.

Come on Baby, Light My Fire: Sparking Further Research in Socio-Affective Mechanisms of Music Using Computational Advancements

“…Computational emotion recognition using facial cues has been incorporated into the study of empathic behavior in group settings. For instance, Kumano et al (2011Kumano et al ( , 2014Kumano et al ( , 2015) conducted a series of experiments to see if empathic interactions could be predicted based on facial data from video recordings of four-person meetings. Their Naive Bayes Network Model was able to predict empathy state given facial expression information across time and improved when parameters such as reaction time in mirrored expression between interlocutors and head gesture annotations were added.…”

Computational Analysis and Simulation of Empathic Behaviors: a Survey of Empathy Modeling with Behavioral Signal Processing Framework

“…Kumano et al extended this framework by investigating reaction timing and facial expression congruence information [36]. They demonstrated that these two aspects were related to the annotated empathy labels ( e.g., a congruent but delayed reaction in facial expression is less likely to have an empathy label).…”

“…Existing works have pulled audio recordings from a few large scale psychotherapy studies totaling to thousands of sessions [4,55,39,40,67,72], however, only a small fraction was finely annotated — in terms of both psychological assessments of mental and behavioral states, and having time-marked transcripts to train and validate automatic speech and language processing systems. The work by Kumano et al has employed a small dataset totaling to a few hours [37,36,35,38]. …”