A Multimodal Dataset for Robot Learning to Imitate Social Human-Human Interaction

Tuyen, Nguyen Tan Viet; Georgescu, Alexandra L.; Giulio, Irene Di; Çeliktutan, Oya

doi:10.1145/3568294.3580080

Cited by 5 publications

(1 citation statement)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These studies aim to create precise predictive models at a reduced cost by leveraging human knowledge and experience. Recently, with large datasets becoming publicly available (Singh and Vishwakarma, 2019; Srivastava et al, 2022; Tuyen et al, 2023), data-driven learning techniques have gained traction. For example, Rahmatizadeh et al (2018) introduced a recurrent neural network-based architecture for learning multiple manipulation tasks from demonstrations.…”

Section: Introductionmentioning

confidence: 99%

Nonverbal social behavior generation for social robots using end-to-end learning

Ko,

Jang,

Lee

et al. 2023

The International Journal of Robotics Research

View full text Add to dashboard Cite

Social robots facilitate improved human–robot interactions through nonverbal behaviors such as handshakes or hugs. However, the traditional methods, which rely on precoded motions, are predictable and can detract from the perception of robots as interactive agents. To address this issue, we have introduced a Seq2Seq-based neural network model that learns social behaviors from human–human interactions in an end-to-end manner. To mitigate the risk of invalid pose sequences during long-term behavior generation, we incorporated a generative adversarial network (GAN). This proposed method was tested using the humanoid robot, Pepper, in a simulated environment. Given the challenges in assessing the success of social behavior generation, we devised novel metrics to quantify the discrepancy between the generated and ground-truth behaviors. Our analysis reveals the impact of different networks on behavior generation performance and compares the efficacy of learning multiple behaviors versus a single behavior. We anticipate that our method will find application in various sectors, including home service, guide, delivery, educational, and virtual robots, thereby enhancing user interaction and enjoyment.

show abstract

Section: Introductionmentioning

confidence: 99%