Objectively measuring drivers’ emotions in real-world conditions is a challenging endeavour. This study investigated whether drivers’ emotional responses were captured more meaningfully by unimodal measurements or by a multimodal machine-learning approach. Ten participants drove a 23-mile route around Sunnyvale, California, while their heart rate, breathing rate and facial expressions were recorded. At regular intervals, participants indicated how they were feeling. After the study, independent observers reviewed a sample of the videotaped sessions, classifying the participants as experiencing high or low levels of stress according to their behaviour. The degree to which drivers’ self-report scores, single physiological data streams and facial behaviour - as judged by a facial recognition machine classifier - reflected their actual stress levels was compared to the multimodal algorithmic estimates which combined physiological and facial data outputs. The results showed that, compared to the other single data modes, the multimodal approach captured how the participants were feeling in a way which most meaningfully corresponded with their observed behaviour. The findings re-affirm the need for multimodal emotion-recognition systems for capturing driver stress. The limitations of the study are also discussed.

No abstract

By utilizing different communication channels, such as verbal language, gestures or facial expressions, virtually embodied interactive humans hold a unique potential to bridge the gap between human-computer interaction and actual interhuman communication. The use of virtual humans is consequently becoming increasingly popular in a wide range of areas where such a natural communication might be beneficial, including entertainment, education, mental health research and beyond. Behind this development lies a series of technological advances in a multitude of disciplines, most notably natural language processing, computer vision, and speech synthesis. In this paper we discuss a Virtual Human Journalist, a project employing a number of novel solutions from these disciplines with the goal to demonstrate their viability by producing a humanoid conversational agent capable of naturally eliciting and reacting to information from a human user. A set of qualitative and quantitative evaluation sessions demonstrated the technical feasibility of the system whilst uncovering a number of deficits in its capacity to engage users in a way that would be perceived as natural and emotionally engaging. We argue that naturalness should not always be seen as a desirable goal and suggest that deliberately suppressing the naturalness of virtual human interactions, such as by altering its personality cues, might in some cases yield more desirable results.

No abstract

The phenomenon of empathy is often examined in contexts where people demonstrate compassion towards others. However, less is known about the way in which we empathise with others in everyday conversations which are less emotionally-charged. This study sought to explore whether there is a relationship between people's expression levels and the degree to which they are perceived to be empathising. 408 participants were shown thin-slice clips of pairs of people demonstrating varying degrees of empathy towards each other, as rated by groups of naive online observers. The participants then rated the degree to which each interlocutor was behaving in an expressive way. Linear mixed effects modelling was used to test whether there was a significant relationship between the interlocutors' expressivity levels and perceptions of how empathically they were behaving. The results indicated the existence of such a relationship with an increase of 0.14 units in empathic behaviour intensity ratings observed for every increase of 1 unit in expressivity level. A second model explored whether this effect was influenced by the gender dynamic of the interaction. The results revealed that expressivity conveyed empathic understanding and responsivity in female-female and mixed-sex dyads but did not appear to do so for male-male dyads. The findings offer support to the notion that behaving in a highly expressive way can be a way of conveying empathy in everyday conversations, but this effect may be dependent on an empathiser's gender in relation to their target. Further studies with more examples of naturalistic empathising behaviour will help to establish the generalisability of this finding.