Gesture encoding and reproduction for human‐robot interaction in text‐to‐gesture systems

Kim, Heon‐Hui; Ha, Yun-Su; Bien, Zeungnam; Park, Kwang‐Hyun

doi:10.1108/01439911211268705

Cited by 9 publications

(9 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Seventeen (77%) of the studies (n = 22) include metaphoric gestures, combined with other gestures [15]- [17], [28], [31], [34]- [37], [39]- [42], [45]- [48]. Thirteen (59%) of the studies (n = 22) include deictic gestures, combined with other gestures [15]- [17], [28], [31], [34], [37], [39]- [41], [45]- [50]. Lastly, seventeen (77%) studies (n = 22) included beat gestures [15], [16], [28], [31], [34]- [36], [38], [39], [42]- [47], [49], [50].…”

Section: B Embodied Conversational Agentsmentioning

confidence: 99%

“…Thirteen (59%) of the studies (n = 22) include deictic gestures, combined with other gestures [15]- [17], [28], [31], [34], [37], [39]- [41], [45]- [50]. Lastly, seventeen (77%) studies (n = 22) included beat gestures [15], [16], [28], [31], [34]- [36], [38], [39], [42]- [47], [49], [50]. Half of the studies had the ECA perform 'random gestures' that were included in the evaluation (i.e.…”

Section: B Embodied Conversational Agentsmentioning

confidence: 99%

“…Generation Method Objective Metrics Agent [28] Data Driven Variation on Mean Squared Error NAO [44] Rule Based -REEM-C [34] Data Driven -Virtual Agent (3D) [31] Data Driven -Android Erica [43] Data Driven Log-likelihood of generated motion Pepper [39] Data Driven -Virtual Agent (3D) [36] Rule Based -Virtual Agent (3D) [16] Data Driven -Wakamaru [41] Rule Based Qualitative Analysis of Joint Positions ASIMO [42] Rule Based -NAO [37] Data Driven -Virtual Agent (3D) [15] Rule Based -Wakamaru [17] Rule Based -ASIMO [40] Rule Based Qualitative Analysis of Joint Positions ASIMO [50] Rule Based -Industrial Service Robot [45] Rule Based -NAO [47] Rule Based -Nao [48] Rule Based -ASIMO [38] Data Driven Cost Function on Kinematic Parameters Virtual Agent (3D) [46] Rule Based -ASIMO [35] Data Driven -Virtual Agent (3D) [49] Rule Based -Fritz narrow cultural and social viewpoint, and some gestures that are acceptable and natural in other cultures may have been misrepresented or rated poorly in the evaluation process from the use of a more restricted sample. The reported participant sample also came from diverse geographical locations, which is known to have developed unique gesture styles and communication methods in regional areas.…”

Section: Studymentioning

confidence: 99%

See 2 more Smart Citations

A Review of Evaluation Practices of Gesture Generation in Embodied Conversational Agents

Wolfert,

Robinson,

Belpaeme

2021

Preprint

View full text Add to dashboard Cite

Embodied Conversational Agents (ECA) take on different forms, including virtual avatars or physical agents, such as a humanoid robot. ECAs are often designed to produce nonverbal behaviour to complement or enhance its verbal communication. One form of nonverbal behaviour is co-speech gesturing, which involves movements that the agent makes with its arms and hands that is paired with verbal communication. Co-speech gestures for ECAs can be created using different generation methods, such as rule-based and data-driven processes. However, reports on gesture generation methods use a variety of evaluation measures, which hinders comparison. To address this, we conducted a systematic review on co-speech gesture generation methods for iconic, metaphoric, deictic or beat gestures, including their evaluation methods. We reviewed 22 studies that had an ECA with a human-like upper body that used co-speech gesturing in a social human-agent interaction, including a user study to evaluate its performance. We found most studies used a within-subject design and relied on a form of subjective evaluation, but lacked a systematic approach. Overall, methodological quality was low-to-moderate and few systematic conclusions could be drawn. We argue that the field requires rigorous and uniform tools for the evaluation of co-speech gesture systems. We have proposed recommendations for future empirical evaluation, including standardised phrases and test scenarios to test generative models. We have proposed a research checklist that can be used to report relevant information for the evaluation of generative models as well as to evaluate co-speech gesture use.

show abstract

Section: B Embodied Conversational Agentsmentioning

confidence: 99%

Section: B Embodied Conversational Agentsmentioning

confidence: 99%

Section: Studymentioning

confidence: 99%

See 1 more Smart Citation

A Review of Evaluation Practices of Gesture Generation in Embodied Conversational Agents

Wolfert,

Robinson,

Belpaeme

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…A human gesture is decomposed into three phases of motion, namely preparation, stroke, and retraction [17]. The stroke is the fragment of higher gestural effort, whilst the preparation and retraction elements consist of moving the arms to and from the rest position, to and from the start and end of the stroke.…”

Section: Introductionmentioning

confidence: 99%

Part-of-Speech and Prosody-based Approaches for Robot Speech and Gesture Synchronization

Pérez-Mayos¹,

Farrús²,

Segura³

2019

J Intell Robot Syst

View full text Add to dashboard Cite

Humanoid robots are already among us and they are beginning to assume more social and personal roles, like guiding and assisting people. Thus, they should interact in a human-friendly manner, using not only verbal cues but also synchronized non-verbal and para-verbal cues. However, available robots are not able to communicate in this multimodal way, being just able to perform predefined gesture sequences, handcrafted to accompany specific utterances. In the current paper, we propose a model based on three different approaches to extend humanoid robots communication behaviour with upper body gestures synchronized with the speech for novel utterances, exploiting part-of-speech grammatical information, prosody cues, and a combination of both. User studies confirm that our methods are able to produce natural, appropriate and good timed gesture sequences synchronized with speech, using both beat and emblematic gestures.

show abstract

“…Thus, robots should use available communication channels as human-like as possible to deliver the correct information to its human partner. Most researches done in HRI regarding emotion expression focused on face-based expressions (Sosnowski et al, 2006;Kishi et al, 2012), emotional speech cues (Ariyoshi et al, 2004) or gestures (Kim et al, 2012). However, for a humanoid robot, all those channels are not necessarily available usually due to hardware constraints.…”

Section: Introductionmentioning

confidence: 99%

Designing robot behavior in human robot interaction based on emotion expression

Wei

Zhao

2016

View full text Add to dashboard Cite

Purpose This paper aims to deal with the problem of designing robot behaviors (mainly to robotic arms) to express emotions. The authors study the effects of robot behaviors from our humanoid robot NAO on the subject’s emotion expression in human–robot interaction (HRI). Design/methodology/approach A method to design robot behavior through the movement primitives is proposed. Then, a novel dimensional affective model is built. Finally, the concept of action semantics is adopted to combine the robot behaviors with emotion expression. Findings For the evaluation of this combination, the authors assess positive (excited and happy) and negative (frightened and sad) emotional patterns on 20 subjects which are divided into two groups (whether they were familiar with robots). The results show that the recognition of the different emotion patterns does not have differences between the two groups and the subjects could recognize the robot behaviors with emotions. Practical implications Using affective models to guide robots’ behavior or express their intentions is highly beneficial in human–robot interaction. The authors think about several applications of the emotional motion: improve efficiency in HRI, direct people during disasters, better understanding with human partners or help people perform their tasks better. Originality/value This paper presents a method to design robot behaviors with emotion expression. Meanwhile, a similar methodology can be used in other parts (leg, torso, head and so on) of humanoid robots or non-humanoid robots, such as industrial robots.

show abstract

Gesture encoding and reproduction for human‐robot interaction in text‐to‐gesture systems

Cited by 9 publications

References 28 publications

A Review of Evaluation Practices of Gesture Generation in Embodied Conversational Agents

A Review of Evaluation Practices of Gesture Generation in Embodied Conversational Agents

Part-of-Speech and Prosody-based Approaches for Robot Speech and Gesture Synchronization

Designing robot behavior in human robot interaction based on emotion expression

Contact Info

Product

Resources

About