Engagement Modeling in Dyadic Interaction

Dermouche, Soumia; Pélachaud, Catherine

doi:10.1145/3340555.3353765

Cited by 26 publications

(28 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These low-level signals are processed using EyesWeb and other external tools, such as machine learning pretrained models (Dermouche and Pelachaud, 2019;Wang et al, 2019), to extract high-level features about the user, such as their level of engagement.…”

Section: Dimensions Of Studymentioning

confidence: 99%

“…This article is organized as follows: in Section 2, we review the main theories about adaptation which our work relies on, in particular Burgoon and others' work; in Section 3, we present an overview of existing models that focus on adapting the ECA's behavior according to the user's behavior; in Section 4, we specify the dimensions we focused on in our adaptation models; in Section 5, we present the general architecture we conceived to endow our ECA with the capability of adapting its behavior to the user's reactions in real time; in Section 6, we describe the scenario we conceived to test the different adaptation models; in Sections 7-9, we report the implementation and evaluation of each of the three models. More details about them can be found in our previous articles (Biancardi et al, 2019b;Biancardi et al, 2019a;Dermouche and Pelachaud, 2019). We finally discuss the results of our work and possible improvements in Sections 10, 11, respectively.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Adaptation Mechanisms in Human–Agent Interaction: Effects on User’s Impressions and Engagement

Biancardi

Dermouche

Pélachaud

2021

Front. Comput. Sci.

Self Cite

View full text Add to dashboard Cite

Adaptation is a key mechanism in human–human interaction. In our work, we aim at endowing embodied conversational agents with the ability to adapt their behavior when interacting with a human interlocutor. With the goal to better understand what the main challenges concerning adaptive agents are, we investigated the effects on the user’s experience of three adaptation models for a virtual agent. The adaptation mechanisms performed by the agent take into account the user’s reaction and learn how to adapt on the fly during the interaction. The agent’s adaptation is realized at several levels (i.e., at the behavioral, conversational, and signal levels) and focuses on improving the user’s experience along different dimensions (i.e., the user’s impressions and engagement). In our first two studies, we aim to learn the agent’s multimodal behaviors and conversational strategies to dynamically optimize the user’s engagement and impressions of the agent, by taking them as input during the learning process. In our third study, our model takes both the user’s and the agent’s past behavior as input and predicts the agent’s next behavior. Our adaptation models have been evaluated through experimental studies sharing the same interacting scenario, with the agent playing the role of a virtual museum guide. These studies showed the impact of the adaptation mechanisms on the user’s experience of the interaction and their perception of the agent. Interacting with an adaptive agent vs. a nonadaptive agent tended to be more positively perceived. Finally, the effects of people’s a priori about virtual agents found in our studies highlight the importance of taking into account the user’s expectancies in human–agent interaction.

show abstract

Section: Dimensions Of Studymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Adaptation Mechanisms in Human–Agent Interaction: Effects on User’s Impressions and Engagement

Biancardi

Dermouche

Pélachaud

2021

Front. Comput. Sci.

Self Cite

View full text Add to dashboard Cite

show abstract

“…The agent that adapted its behavior to maximize user's engagement was perceived as warm by participants, but they did not find any effect of agent's adaptation on users' evaluation of their experience of the interaction. As noted in Dermouche and Pelachaud (2019a), engagement was defined by user's behaviors that included gaze directions, facial expressions, and posture shifts. Bickmore et al (2011) found that the use of relational behavior lead to significantly greater engagement by museum visitors.…”

Section: Adaptation Mechanismsmentioning

confidence: 99%

“…LSTM are recurrent neural networks able to capture the different dynamics of time series and they have been shown to be efficient in sequence prediction problems. These models have been successfully applied to engagement recognition using head movements in Hadfield et al ( 2018 ) and Lala et al ( 2017 ) and facial expression in Dermouche and Pelachaud ( 2019a ). Temporal models such as LSTM and Gated Recurrent Unit (GRU) are compared to static deep leaning approaches as well as logistic regression.…”

Section: Perceptionmentioning

confidence: 99%

Engagement in Human-Agent Interaction: An Overview

et al. 2020

Self Cite

View full text Add to dashboard Cite

Engagement is a concept of the utmost importance in human-computer interaction, not only for informing the design and implementation of interfaces, but also for enabling more sophisticated interfaces capable of adapting to users. While the notion of engagement is actively being studied in a diverse set of domains, the term has been used to refer to a number of related, but different concepts. In fact it has been referred to across different disciplines under different names and with different connotations in mind. Therefore, it can be quite difficult to understand what the meaning of engagement is and how one study relates to another one accordingly. Engagement has been studied not only in human-human, but also in human-agent interactions i.e., interactions with physical robots and embodied virtual agents. In this overview article we focus on different factors involved in engagement studies, distinguishing especially between those studies that address task and social engagement, involve children and adults, are conducted in a lab or aimed for long term interaction. We also present models for detecting engagement and for generating multimodal behaviors to show engagement.

show abstract

“…In social robotics, engagement can be defined as "the process by which two (or more) participants establish, maintain, and end their perceived connection to one another" [1]. Aside from few attempts in human-human interaction [2] and in human-virtual agent interaction [3,4] at integrating secondparty information in the analysis of user's socio-emotional behavior, current human-robot interaction (HRI) systems rely only on the user data without exploiting the contextual information offered by the robot data, despite the evident link between the robot's behavior and the user's socio-emotional state. We argue that architectures for automatic user engagement analysis can benefit from using the robot data, as well as from learning the interaction dynamics between the user and the robot.…”

Section: Introductionmentioning

confidence: 99%

HRI-RNN: A User-Robot Dynamics-Oriented RNN for Engagement Decrease Detection

Atamna¹,

Clavel²

2020

Interspeech 2020

View full text Add to dashboard Cite

Natural and fluid human-robot interaction (HRI) systems rely on the robot's ability to accurately assess the user's engagement in the interaction. Current HRI systems for engagement analysis, and more broadly emotion recognition, only consider user data while discarding robot data which, in many cases, affects the user state. We present a novel recurrent neural architecture for online detection of user engagement decrease in a spontaneous HRI setting that exploits the robot data. Our architecture models the user as a distinct party in the conversation and uses the robot data as contextual information to help assess engagement. We evaluate our approach on a real-world highly imbalanced data set, where we observe up to 2.13% increase in F1 score compared to a standard gated recurrent unit (GRU).

show abstract

Engagement Modeling in Dyadic Interaction

Cited by 26 publications

References 16 publications

Adaptation Mechanisms in Human–Agent Interaction: Effects on User’s Impressions and Engagement

Adaptation Mechanisms in Human–Agent Interaction: Effects on User’s Impressions and Engagement

Engagement in Human-Agent Interaction: An Overview

HRI-RNN: A User-Robot Dynamics-Oriented RNN for Engagement Decrease Detection

Contact Info

Product

Resources

About