Dynamic Future Net

Chen, Wenheng; Wang, He; Shao, Tianjia; Zhou, Kun

doi:10.1145/3394171.3413669

Cited by 19 publications

(2 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, it is expected that the performance can be further improved through attempts such as using the transformer-based model proposed in Vaswani et al (2017). Finally, we would like to conduct further research on a one-to-many generative model so that a single human behavior can be mapped to multiple robot responses by referring to Sun et al (2020), Chen et al (2020), Yu and Tapus (2020), and other studies. If the robot can learn various behavioral policies considering the user’s age or cultural differences, more natural human–robot interaction can be obtained.…”

Section: Discussionmentioning

confidence: 99%

Nonverbal social behavior generation for social robots using end-to-end learning

Ko,

Jang,

Lee

et al. 2023

The International Journal of Robotics Research

View full text Add to dashboard Cite

Social robots facilitate improved human–robot interactions through nonverbal behaviors such as handshakes or hugs. However, the traditional methods, which rely on precoded motions, are predictable and can detract from the perception of robots as interactive agents. To address this issue, we have introduced a Seq2Seq-based neural network model that learns social behaviors from human–human interactions in an end-to-end manner. To mitigate the risk of invalid pose sequences during long-term behavior generation, we incorporated a generative adversarial network (GAN). This proposed method was tested using the humanoid robot, Pepper, in a simulated environment. Given the challenges in assessing the success of social behavior generation, we devised novel metrics to quantify the discrepancy between the generated and ground-truth behaviors. Our analysis reveals the impact of different networks on behavior generation performance and compares the efficacy of learning multiple behaviors versus a single behavior. We anticipate that our method will find application in various sectors, including home service, guide, delivery, educational, and virtual robots, thereby enhancing user interaction and enjoyment.

show abstract

Section: Discussionmentioning

confidence: 99%

Nonverbal social behavior generation for social robots using end-to-end learning

Ko,

Jang,

Lee

et al. 2023

The International Journal of Robotics Research

View full text Add to dashboard Cite

show abstract

“…This model consists of an RNN (comprised of LSTM cells) for motion synthesis combined with an adversarial neural network (similar to a GAN) for "refining" the produced motion (control) so as to be identical to the reference input motion. A new deep learning model, called Dynamic Future Net (DFN) [72], was developed to produce diversified motion (walking-torunning, walking-to-dancing, etc.) with arbitrary duration, given a short-length pose sequence (e.g.…”

Section: Diversified Motion Synthesismentioning

confidence: 99%

Machine Learning Approaches for 3D Motion Synthesis and Musculoskeletal Dynamics Estimation: A Survey

Loi,

Zacharaki,

Moustakas

2024

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

The inference of 3D motion and dynamics of the human musculoskeletal system has traditionally been solved using physics-based methods that exploit physical parameters to provide realistic simulations. Yet, such methods suffer from computational complexity and reduced stability, hindering their use in computer graphics applications that require real-time performance. With the recent explosion of data capture (mocap, video) machine learning (ML) has started to become popular as it is able to create surrogate models harnessing the huge amount of data stemming from various sources, minimizing computational time (instead of resource usage), and most importantly, approximate real-time solutions. The main purpose of this paper is to provide a review and classification of the most recent works regarding motion prediction, motion synthesis as well as musculoskeletal dynamics estimation problems using ML techniques, in order to offer sufficient insight into the state-of-the-art and draw new research directions. While the study of motion may appear distinct to musculoskeletal dynamics, these application domains provide jointly the link for more natural computer graphics character animation, since ML-based musculoskeletal dynamics estimation enables modeling of more long-term, temporally evolving, ergonomic effects, while offering automated and fast solutions. Overall, our review offers an in-depth presentation and classification of ML applications in human motion analysis, unlike previous survey articles focusing on specific aspects of motion prediction.

show abstract