Topic Transition Strategies for an Information-Giving Agent

Glas, Nadine; Pélachaud, Catherine

doi:10.18653/v1/w15-4725

Cited by 6 publications

(5 citation statements)

References 16 publications

(14 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…First, it must be valid to both human-agent and human-human interactions. This prohibits using the term "user" in the definition itself, even though one can mention it when the context allows it (Glas and Pelachaud, 2015b). This also incites not to consider engagement only on the basis of having the interaction with an ECA going on for as long as possible (Oertel et al, 2020).…”

Section: Ground Rules For a Proper Definition Of Engagementmentioning

confidence: 99%

A multimodal approach for modeling engagement in conversation

Pellet-Rostaing

Bertrand²,

Boudin³

et al. 2023

Front. Comput. Sci.

View full text Add to dashboard Cite

Recently, engagement has emerged as a key variable explaining the success of conversation. In the perspective of human-machine interaction, an automatic assessment of engagement becomes crucial to better understand the dynamics of an interaction and to design socially-aware robots. This paper presents a predictive model of the level of engagement in conversations. It shows in particular the interest of using a rich multimodal set of features, outperforming the existing models in this domain. In terms of methodology, study is based on two audio-visual corpora of naturalistic face-to-face interactions. These resources have been enriched with various annotations of verbal and nonverbal behaviors, such as smiles, head nods, and feedbacks. In addition, we manually annotated gestures intensity. Based on a review of previous works in psychology and human-machine interaction, we propose a new definition of the notion of engagement, adequate for the description of this phenomenon both in natural and mediated environments. This definition have been implemented in our annotation scheme. In our work, engagement is studied at the turn level, known to be crucial for the organization of the conversation. Even though there is still a lack of consensus around their precise definition, we have developed a turn detection tool. A multimodal characterization of engagement is performed using a multi-level classification of turns. We claim a set of multimodal cues, involving prosodic, mimo-gestural and morpho-syntactic information, is relevant to characterize the level of engagement of speakers in conversation. Our results significantly outperform the baseline and reach state-of-the-art level (0.76 weighted F-score). The most contributing modalities are identified by testing the performance of a two-layer perceptron when trained on unimodal feature sets and on combinations of two to four modalities. These results support our claim about multimodality: combining features related to the speech fundamental frequency and energy with mimo-gestural features leads to the best performance.

show abstract

Section: Ground Rules For a Proper Definition Of Engagementmentioning

confidence: 99%

A multimodal approach for modeling engagement in conversation

Pellet-Rostaing

Bertrand²,

Boudin³

et al. 2023

Front. Comput. Sci.

View full text Add to dashboard Cite

show abstract

“…In general a distinction can be made between rulebased and machine learning-based approaches for the prediction of engagement. Examples of studies that report rule-based approaches are , Glas et al (2015), Ishii and Nakano (2008), and Rich et al (2010). There are differences in how rules are implemented.…”

Section: Automatic Prediction Of Engagementmentioning

confidence: 99%

Engagement in Human-Agent Interaction: An Overview

et al. 2020

Self Cite

View full text Add to dashboard Cite

Engagement is a concept of the utmost importance in human-computer interaction, not only for informing the design and implementation of interfaces, but also for enabling more sophisticated interfaces capable of adapting to users. While the notion of engagement is actively being studied in a diverse set of domains, the term has been used to refer to a number of related, but different concepts. In fact it has been referred to across different disciplines under different names and with different connotations in mind. Therefore, it can be quite difficult to understand what the meaning of engagement is and how one study relates to another one accordingly. Engagement has been studied not only in human-human, but also in human-agent interactions i.e., interactions with physical robots and embodied virtual agents. In this overview article we focus on different factors involved in engagement studies, distinguishing especially between those studies that address task and social engagement, involve children and adults, are conducted in a lab or aimed for long term interaction. We also present models for detecting engagement and for generating multimodal behaviors to show engagement.

show abstract

“…Dialogue strategy can also be adapted to user engagement. Topic selection based on user engagement was proposed [45]. The system was designed to predict user engagement on each topic, and select the next topic which maximizes both user engagement and the system's preference.…”

Section: Related Workmentioning

confidence: 99%

Engagement recognition by a latent character model based on multimodal listener behaviors in spoken dialogue

Inoue

Lala

Takanashi

et al. 2018

SIP

View full text Add to dashboard Cite

Engagement represents how much a user is interested in and willing to continue the current dialogue. Engagement recognition will provide an important clue for dialogue systems to generate adaptive behaviors for the user. This paper addresses engagement recognition based on multimodal listener behaviors of backchannels, laughing, head nodding, and eye gaze. In the annotation of engagement, the ground-truth data often differs from one annotator to another due to the subjectivity of the perception of engagement. To deal with this, we assume that each annotator has a latent character that affects his/her perception of engagement. We propose a hierarchical Bayesian model that estimates both engagement and the character of each annotator as latent variables. Furthermore, we integrate the engagement recognition model with automatic detection of the listener behaviors to realize online engagement recognition. Experimental results show that the proposed model improves recognition accuracy compared with other methods which do not consider the character such as majority voting. We also achieve online engagement recognition without degrading accuracy.

show abstract

Topic Transition Strategies for an Information-Giving Agent

Cited by 6 publications

References 16 publications

A multimodal approach for modeling engagement in conversation

A multimodal approach for modeling engagement in conversation

Engagement in Human-Agent Interaction: An Overview

Engagement recognition by a latent character model based on multimodal listener behaviors in spoken dialogue

Contact Info

Product

Resources

About