Incremental Coordination: Attention-Centric Speech Production in a Physically Situated Conversational Agent

Yu, Zhou; Bohus, Dan; Horvitz, Eric

doi:10.18653/v1/w15-4652

Cited by 22 publications

(14 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If the user does not have the turn, the robot either has or takes the turn through its Robot's initiative sub-tree, and executes the presentation. Firstly, joint attention is ensured or grabbed (see Yu et al (2015)) if lost, this can be sensed in multiple ways (Ba and Odobez, 2009;Sheikhi, 2014;Szafir and Mutlu, 2012).…”

Section: Modelling the Presentationmentioning

confidence: 99%

Modelling Adaptive Presentations in Human-Robot Interaction using Behaviour Trees

Axelsson¹,

Skantze²

2019

Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue

View full text Add to dashboard Cite

In dialogue, speakers continuously adapt their speech to accommodate the listener, based on the feedback they receive. In this paper, we explore the modelling of such behaviours in the context of a robot presenting a painting. A Behaviour Tree is used to organise the behaviour on different levels, and allow the robot to adapt its behaviour in real-time; the tree organises engagement, joint attention, turn-taking, feedback and incremental speech processing. An initial implementation of the model is presented, and the system is evaluated in a user study, where the adaptive robot presenter is compared to a non-adaptive version. The adaptive version is found to be more engaging by the users, although no effects are found on the retention of the presented material.

show abstract

Section: Modelling the Presentationmentioning

confidence: 99%

Modelling Adaptive Presentations in Human-Robot Interaction using Behaviour Trees

Axelsson¹,

Skantze²

2019

Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue

View full text Add to dashboard Cite

show abstract

“…The focus of studies based on these definitions was when and how the conversation starts and also ends. For example, one of the tasks was to detect the engaged person who wants to start the conversation with a situated robot [28,29]. The second type of definition is about the quality of the connection between participants during the dialogue.…”

Section: A) Definition Of Engagementmentioning

confidence: 99%

Engagement recognition by a latent character model based on multimodal listener behaviors in spoken dialogue

Inoue

Lala

Takanashi

et al. 2018

SIP

View full text Add to dashboard Cite

Engagement represents how much a user is interested in and willing to continue the current dialogue. Engagement recognition will provide an important clue for dialogue systems to generate adaptive behaviors for the user. This paper addresses engagement recognition based on multimodal listener behaviors of backchannels, laughing, head nodding, and eye gaze. In the annotation of engagement, the ground-truth data often differs from one annotator to another due to the subjectivity of the perception of engagement. To deal with this, we assume that each annotator has a latent character that affects his/her perception of engagement. We propose a hierarchical Bayesian model that estimates both engagement and the character of each annotator as latent variables. Furthermore, we integrate the engagement recognition model with automatic detection of the listener behaviors to realize online engagement recognition. Experimental results show that the proposed model improves recognition accuracy compared with other methods which do not consider the character such as majority voting. We also achieve online engagement recognition without degrading accuracy.

show abstract

“…To realize smooth interaction, it is essential for the system to correctly identify who talks to whom and if the user is giving his/her attention toward ERICA (Yu et al, 2015). In this demonstration, we track the user's location and head orientation in the 3D space by using the Kinect v2 sensor.…”

Section: Speaker Tracking With Depth Cameramentioning

confidence: 99%

Talking with ERICA, an autonomous android

Inoue¹,

Milhorat²,

Lala³

et al. 2016

Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue

View full text Add to dashboard Cite

We demonstrate dialogues with an autonomous android ERICA, who has an appearance like a human being. Currently, ERICA plays two social roles: a laboratory guide and a counselor. It is designed to follow the protocols of human dialogue to make the user comfortable: (1) having a chat before the main talk, (2) proactively asking questions, and (3) conveying proper feedbacks. The combination of the human-like appearance and the appropriate behaviors according to her social roles allows for symbiotic human-robot interaction.

show abstract

Incremental Coordination: Attention-Centric Speech Production in a Physically Situated Conversational Agent

Cited by 22 publications

References 10 publications

Modelling Adaptive Presentations in Human-Robot Interaction using Behaviour Trees

Modelling Adaptive Presentations in Human-Robot Interaction using Behaviour Trees

Engagement recognition by a latent character model based on multimodal listener behaviors in spoken dialogue

Talking with ERICA, an autonomous android

Contact Info

Product

Resources

About