Learning predict-and-simulate policies from unorganized human motion data

Abstract

The goal of this research is to create physically simulated biped characters equipped with a rich repertoire of motor skills. The user can control the characters interactively by modulating their control objectives. The characters can interact physically with each other and with the environment. We present a novel network-based algorithm that learns control policies from unorganized, minimally-labeled human motion data. The network architecture for interactive character animation incorporates an RNN-based motion generator into a DRL-based controller for physics simulation and control. The motion generator guides forward dynamics simulation by feeding a sequence of future motion frames to track. The rich future prediction facilitates policy learning from large training data sets. We will demonstrate the effectiveness of our approach with biped characters that learn a variety of dynamic motor skills from large, unorganized data and react to unexpected perturbation beyond the scope of the training data.

ALLSTEPS: Curriculum‐driven Learning of Stepping Stone Skills

“…We choose PPO because it is simple to implement and effective for producing high quality locomotion solutions, as demonstrated in previous work, e.g. [PALvdP18, YTL18, PRL*19, WL19].…”

“…In many cases, these aim to satisfy an imitation objective and target motions on flat terrain, e.g. [LPY16, PBYVDP17, LH17, PRL*19, BCHF19]. Other solutions learn in the absence of motion capture data, also for flat terrain, e.g., [YTL18, LPLL19, JVWDGL19].…”

“…Reverse curriculum learning has been shown to be effective at balancing uneven data generation in DRL. For example, [WL19] and [PRL*19] propose a form of adaptive sampling where more difficult tasks are given higher priority during training.…”

Linear Time Stable PD Controllers for Physics‐based Character Animation

“…The most straight forward SPD implementation uses direct matrix inversion to solve the system [LGH*18, LPLL19, PRL*19]. This approach takes O ( n 3 ) time and may encounter numerical stability issues.…”

“…Physics‐based character animation has made significant progresses in recent years. High quality controllers and skills can now be automatically learned to generate motions that are indistinguishable from motion capture data in real‐time [YTL18, PALvdP18, PRL*19, BCHF19]. Physics‐based characters are yet to be widely adopted by the industry, however, as simulation times for controllable characters still remain as one of the bottlenecks.…”