Effects of Emotional Expressiveness of a Female Digital Human on Loneliness, Stress, Perceived Support, and Closeness Across Genders: Randomized Controlled Trial

Abstract

Background Loneliness is a growing public health problem that has been exacerbated in vulnerable groups during the COVID-19 pandemic. Social support interventions have been shown to reduce loneliness, including when delivered through technology. Digital humans are a new type of computer agent that show promise as supportive peers in health care. For digital humans to be effective and engaging support persons, it is important that they develop closeness with people. Closeness can be increased by emotional expressiveness, particularly in female relationships. However, it is unknown whether emotional expressiveness improves relationships with digital humans and affects physiological responses. Objective The aim of this study is to investigate whether emotional expression by a digital human can affect psychological and physiological outcomes and whether the effects are moderated by the user’s gender. Methods A community sample of 198 adults (101 women, 95 men, and 2 gender-diverse individuals) was block-randomized by gender to complete a 15-minute self-disclosure conversation with a female digital human in 1 of 6 conditions. In these conditions, the digital human varied in modality richness and emotional expression on the face and in the voice (emotional, neutral, or no face; emotional or neutral voice). Perceived loneliness, closeness, social support, caring perceptions, and stress were measured after each interaction. Heart rate, skin temperature, and electrodermal activity were assessed during each interaction. 3-way factorial analyses of variance with post hoc tests were conducted. Results Emotional expression in the voice was associated with greater perceptions of caring and physiological arousal during the interaction, and unexpectedly, with lower feelings of support. User gender moderated the effect of emotional expressiveness on several outcomes. For women, an emotional voice was associated with increased closeness, social support, and caring perceptions, whereas for men, a neutral voice increased these outcomes. For women, interacting with a neutral face was associated with lower loneliness and subjective stress compared with no face. Interacting with no face (ie, a voice-only black screen) resulted in lower loneliness and subjective stress for men, compared with a neutral or emotional face. No significant results were found for heart rate or skin temperature. However, average electrodermal activity was significantly higher for men while interacting with an emotional voice. Conclusions Emotional expressiveness in a female digital human has different effects on loneliness, social, and physiological outcomes for men and women. The results inform the design of digital human support persons and have theoretical implications. Further research is needed to evaluate how more pronounced emotional facial expressions in a digital human might affect the results. Trial Registration Australia New Zealand Clinical Trials Registry (ANZCTR) ACTRN12621000865819; https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=381816&isReview

Talk it Out: Teen Reactions to a Supportive Conversational Agent

“…Participants commented positively when they felt the conversation flow/experience felt similar to a natural conversation. Participants' preference for emotionally expressive voice is confirmed by prior studies, for example Loveys et al (2021) who showed that emotional expressiveness in the voice resulted in increased feelings of caring (in female participants). Negative comments generally revolved around perceptions of an impersonal or robotic nature of the conversational agent, inappropriate pace of conversation (too slow or fast), too few conversational turns, and dislike of some interaction topics.…”

A Neuro-Computational Model for Discrete-Continuous Dual-Task Process

“…The on-off term could describe different performances that the units are involved in and characterize the tasks' nature, since each task requires a specific amount of attention (Castellanos et al, 2005 ; Ghorbanian et al, 2013 ; Katsuki and Constantinidis, 2013 ). The a AT and a PT parameters are related to the variability of the performance, since variability is counted as the attentional diversion indicator in various studies (Voss et al, 2004 ; Thissen et al, 2014 ; Loveys et al, 2021 ). Therefore, the value of a AT and a PT is estimated based on the scale of the variance of each task in different sequences.…”

Emotionally intelligent virtual tour guide in handling group conflicts: Effect on user outcomes

“…Abdollahi et al 19 designed a socially assistive robot with empathy, adopting a multimodal emotional recognition algorithm and emotional expression system to provide companionship for the elderly with depression and dementia. The second term “Closeness” refers to personal intimacy and emotional attachment 8 . Loveys et al 5 explored the emotional expression of virtual agents' faces and voices and presented that the presence of face and emotional voice help to establish positive relationships.…”

“…Virtual agents provide positive and catalytic effects on user engagement and immersion with their personalized, interactive, and diversified content. The media equation theory proposes that people respond to computers similarly to how they respond to real people 8 . Thus, humans are likely to interact and perceive virtual agents under the same paradigm.…”