Multimodal Human-Robot Interaction Framework for a Personal Robot

Gorostiza, Javi F.; Barber, Ramón; Khamis, Alaa; Pacheco, Matheus M.; Rivas, Rafael; Corrales, A.; Delgado, Elier Méndez; Salichs, Miguel Á.

doi:10.1109/roman.2006.314392

Cited by 49 publications

(35 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Only a few approaches in robotics incorporate both speech and gesture synthesis; however, in most cases the robots are equipped with a set of pre-recorded gestures that are not generated on-line but simply replayed during humanrobot interaction, as seen in [8] or [40]. Moreover, a majority of approaches focusing on gesture synthesis for humanoid robots are limited to the implementation and evaluation of a single type of gesture, typically deictic (e.g., [32,41]) or emblematic gestures (e.g., [13]) instead of providing a general framework that can handle all types of gesture.…”

Section: Roboticsmentioning

confidence: 99%

Generation and Evaluation of Communicative Robot Gesture

Salem¹,

Kopp

Wachsmuth

et al. 2012

Int J of Soc Robotics

143

View full text Add to dashboard Cite

How is communicative gesture behavior in robots perceived by humans? Although gesture is crucial in social interaction, this research question is still largely unexplored in the field of social robotics. Thus, the main objective of the present work is to investigate how gestural machine behaviors can be used to design more natural communication in social robots. The chosen approach is twofold. Firstly, the technical challenges encountered when implementing a speech-gesture generation model on a robotic platform are tackled. We present a framework that enables the humanoid robot to flexibly produce synthetic speech and co-verbal hand and arm gestures at run-time, while not being limited to a predefined repertoire of motor actions. Secondly, the achieved flexibility in robot gesture is exploited in controlled experiments. To gain a deeper understanding of how communicative robot gesture might impact and shape human perception and evaluation of human-robot interaction, we conducted a between-subjects experimental study using the humanoid robot in a joint task scenario. We manipulated the non-verbal behaviors of the robot in three experimental conditions, so that it would refer to objects by utilizing either (1) unimodal (i.e., speech only) utterances, (2) congruent multimodal (i.e., semantically matching speech and gesture) or (3) incongruent multimodal (i.e., semantically non-matching speech and gesture) utterances. Our findings reveal that the robot is evaluated more positively when nonverbal behaviors such as hand and arm gestures are displayed along with speech, even if they do not semantically match the spoken utterance.

show abstract

Section: Roboticsmentioning

confidence: 99%

Generation and Evaluation of Communicative Robot Gesture

Salem¹,

Kopp

Wachsmuth

et al. 2012

Int J of Soc Robotics

143

View full text Add to dashboard Cite

show abstract

“…Meanwhile, there are some other sociable robots, which were designed to establish relationships with humans, for example, Autom (Kidd and Breazeal, 2008), which was designed to play a role as a weight loss coach to guide humans to loss their weight, which showed that developing long-term close relationship between human and robot was practicable, ATR's Robovie (Kahn et al 2008), which had a social characteristic that could provide physical intimacy (e.g., holding, touching, embracing) with human, and MAGGIE (Gorostiza et al 2006), whose main goal is to interact in a natural way and establish a peer-to-peer relationship with humans. Moreover, there was a research platform for studying social robotic user-interfaces, called iCat (-interactive cat‖), developed by Philips, which was equipped with the capabilities of vision, conversation and facial expression and used to examine the influence of the perceived social abilities of a robot on user's attitude towards and acceptance of the robot (Heerink et al 2006).…”

Section: Robot Attributesmentioning

confidence: 99%

Contextual Recognition of Robot Emotions

Zhang

Sharkey

2011

Lecture Notes in Computer Science

View full text Add to dashboard Cite

In the field of human-robot interaction, socially interactive robots are often equipped with the ability to detect the affective states of users, the ability to express emotions through the use of synthetic facial expressions, speech and textual content, and the ability for imitating and social learning. Past work on creating robots that can make convincing emotional expressions has concentrated on the quality of those expressions, and on assessing people's ability to recognize them. Previous recognition studies presented the facial expressions of the robots in neutral contexts, without any strong emotional valence (e.g., emotionally valenced music or video). It is therefore worth empirically exploring whether observers' judgments of the facial cues of a robot would be affected by a surrounding emotional context. This thesis takes its inspiration from the contextual effects found on the interpretation of the expressions on human faces and computer avatars, and looks at the extent to which they also apply to the interpretation of the facial expressions of a mechanical robot head. The kinds of contexts that affect the recognition of robot emotional expressions, the circumstances under which such contextual effects occur, and the relationship between emotions and the surrounding situation, are observed and analyzed in a series of 11 experiments. In these experiments, the FACS (Facial Action Coding System) (Ekman and Friesen, 2002) was applied to set up the parameters of the servos to make the robot head produce sequences of facial expressions. Four different emotional surrounding or preceding contexts were used (i.e., recorded BBC News pieces, selected affective pictures, classical music pieces and film clips). This thesis provides evidence that observers' judgments about the facial expressions of a robot can be affected by a surrounding emotional context. From a psychological perspective, the contextual effects found on the robotic facial expressions based on the FACS, indirectly support the claims that human emotions are both 4 biologically based and socially constructed. From a robotics perspective, it is argued that the results obtained from the analyses will be useful for guiding researchers to enhance the expressive skills of emotional robots in a surrounding emotional context. This thesis also analyzes the possible factors contributing to the contextual effects found in the original 11 experiments.Some future work, including four new experiments (a preliminary experiment designed to identify appropriate contextual materials and three further experiments in which factors likely to affect a context effect are controlled one by one) is also proposed in this thesis.

show abstract

“…The set of all these robot's skills is in constant growing. More details about Maggie's features and some of her capabilities can be found in [19], [21].…”

Section: Maggie: the Social Robotmentioning

confidence: 99%

End-user programming of a social robot by dialog

Gorostiza

Salichs

2011

Robotics and Autonomous Systems

Self Cite

View full text Add to dashboard Cite

One of the main challenges faced by social robots is how to provide intuitive, natural and enjoyable usability for the end-user. In our ordinary environment, social robots could be important tools for education and entertainment (edutainment) in a variety of ways. This paper presents a Natural Programming System (NPS) that is geared to non-expert users. The main goal of such a system is to provide an enjoyable interactive platform for the users to build different programs within their social robot platform. The end-user can build a complex net of actions and conditions (a sequence) in a social robot via mixed-initiative dialogs and multimodal interaction. The system has been implemented and tested in Maggie, a real social robot with multiple skills, conceived as a general HRI researching platform. The robot's internal features (skills) have been implemented to be verbally accessible to the end-user, who can combine them into others more complex following a bottom-up model. The built sequence is internally implemented as a Sequence Function Chart (SFC), which allows parallel execution, modularity and re-use. A multimodal Dialog Manager System (DMS) takes charge of keeping the coherence of the interaction. This work is thought for bringing social robots closer to non-expert users, who can play the game of "teaching how to do things" with the robot.

show abstract

Multimodal Human-Robot Interaction Framework for a Personal Robot

Cited by 49 publications

References 8 publications

Generation and Evaluation of Communicative Robot Gesture

Generation and Evaluation of Communicative Robot Gesture

Contextual Recognition of Robot Emotions

End-user programming of a social robot by dialog

Contact Info

Product

Resources

About