Multimodal conversational interaction with a humanoid robot

Csapó, Ádám; Gilmartin, Emer; Grizou, Jonathan; Han, Jian; Meena, Raveesh; Anastasiou, Dimitra; Jokinen, Kristiina; Wilcock, Graham

doi:10.1109/coginfocom.2012.6421935

Cited by 40 publications

(14 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, we also experimented with motion capture technology using Kinect as one of the robot's inputs. Preliminary work on this is presented in [2].…”

Section: Discussion and Future Workmentioning

confidence: 99%

“…Further details are given in [2]. The basic technologies, including WikiTalk, face recognition, gesturing and dialogue management, listed in Figure 5, were integrated into the Nao system.…”

Section: Implementation and Evaluationmentioning

confidence: 99%

“…The evaluation is reported in [2] and [14]. Following the evaluation scheme proposed in [8], the users were asked to fill in a questionnaire twice, first to capture their expectations of the system before interacting with it, and then to measure their experience of the system after their interaction with it.…”

Section: Implementation and Evaluationmentioning

confidence: 99%

See 2 more Smart Citations

Multimodal Open-Domain Conversations with the Nao Robot

Jokinen

Wilcock

2013

Natural Interaction With Robots, Knowbots and Smartphones

Self Cite

View full text Add to dashboard Cite

In this paper we discuss the design of human-robot interaction focussing especially on social robot communication and multimodal information presentation. As a starting point we use the WikiTalk application, an open-domain conversational system which has been previously developed using a robotics simulator. We describe how it can be implemented on the Nao robot platform, enabling Nao to make informative spoken contributions on a wide range of topics during conversation. Spoken interaction is further combined with gesturing in order to support Nao's presentation by natural multimodal capabilities, and to enhance and explore natural communication between human users and robots.

show abstract

“…Finally, we also experimented with motion capture technology using Kinect as one of the robot's inputs. Preliminary work on this is presented in [2].…”

Section: Discussion and Future Workmentioning

confidence: 99%

“…Further details are given in [2]. The basic technologies, including WikiTalk, face recognition, gesturing and dialogue management, listed in Figure 5, were integrated into the Nao system.…”

Section: Implementation and Evaluationmentioning

confidence: 99%

See 1 more Smart Citation

Multimodal Open-Domain Conversations with the Nao Robot

Jokinen

Wilcock

2013

Natural Interaction With Robots, Knowbots and Smartphones

Self Cite

View full text Add to dashboard Cite

show abstract

“…An interesting approach is presented in [10], in which a multimodal conversational interaction system is developed which enhances the naturalness and expressivity of interactions. A brief overview of some key aspects, issues and challenges of multimodal interaction is given in [11].…”

Section: Multimodal Interfaces In Hrimentioning

confidence: 99%

Point-and-Command Paradigm for Interaction with Assistive Robots

Boboc

Dumitru

Antonya

2015

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

This paper presents a multi-modal interface for the interaction between a human user and an assistive humanoid robot. The interaction is performed through gestural commands and a dialogue mechanism to provide a 'natural' means for the user to command an assistive robot to perform several given tasks. A "Point-and-Command" interaction concept was proposed with the aim of providing the user with an easy-to-use and intuitive interface and to improve the efficiency of human-robot collaboration. Likewise, a decision support system (DSS) was implemented based on fuzzy logic, in order to warn the user when an active robot is about to become unusable and should be replaced. Thus, the interaction time, which depends on the robot's battery life, can be increased by replacing the assistant with another more fully charged one, making the interaction seem more natural.

show abstract

“…The task of integrating non-verbal gestures in the Nao humanoid robot was part of a project on multimodal conversational interaction with a humanoid robot [4]. We started with WikiTalk [5], a spoken dialogue system for open domain conversation using Wikipedia as a knowledge source.…”

Section: Gestures and Naomentioning

confidence: 99%

Integration of gestures and speech in human-robot interaction

Meena

Jokinen

Wilcock

2012

2012 IEEE 3rd International Conference on Cognitive Infocommunications (CogInfoCom)

Self Cite

View full text Add to dashboard Cite

Abstract-We present an approach to enhance the interaction abilities of the Nao humanoid robot by extending its communicative behavior with non-verbal gestures (hand and head movements, and gaze following). A set of nonverbal gestures were identified that Nao could use for enhancing its presentation and turn-management capabilities in conversational interactions. We discuss our approach for modeling and synthesizing gestures on the Nao robot. A scheme for system evaluation that compares the values of users' expectations and actual experiences has been presented. We found that open arm gestures, head movements and gaze following could significantly enhance Nao's ability to be expressive and appear lively, and to engage human users in conversational interactions.

show abstract

Multimodal conversational interaction with a humanoid robot

Cited by 40 publications

References 10 publications

Multimodal Open-Domain Conversations with the Nao Robot

Multimodal Open-Domain Conversations with the Nao Robot

Point-and-Command Paradigm for Interaction with Assistive Robots

Integration of gestures and speech in human-robot interaction

Contact Info

Product

Resources

About