Learning locomotion skills using DeepRL

Peng, Xue Bin; Panne, Michiel van de

doi:10.1145/3099564.3099567

Cited by 110 publications

(22 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some of these examples in Imitation Learning are shown in Figure 3. The choice of action space is also shown to have an impact the speed and quality of imitation‐based learning [PvdP17].…”

Section: Skeletal Animationmentioning

confidence: 99%

A Survey on Reinforcement Learning Methods in Character Animation

Kwiatkowski

Alvarado

Kalogeiton

et al. 2022

Computer Graphics Forum

Self Cite

View full text Add to dashboard Cite

Reinforcement Learning is an area of Machine Learning focused on how agents can be trained to make sequential decisions, and achieve a particular goal within an arbitrary environment. While learning, they repeatedly take actions based on their observation of the environment, and receive appropriate rewards which define the objective. This experience is then used to progressively improve the policy controlling the agent's behavior, typically represented by a neural network. This trained module can then be reused for similar problems, which makes this approach promising for the animation of autonomous, yet reactive characters in simulators, video games or virtual reality environments. This paper surveys the modern Deep Reinforcement Learning methods and discusses their possible applications in Character Animation, from skeletal control of a single, physically‐based character to navigation controllers for individual agents and virtual crowds. It also describes the practical side of training DRL systems, comparing the different frameworks available to build such agents.

show abstract

Section: Skeletal Animationmentioning

confidence: 99%

A Survey on Reinforcement Learning Methods in Character Animation

Kwiatkowski

Alvarado

Kalogeiton

et al. 2022

Computer Graphics Forum

Self Cite

View full text Add to dashboard Cite

show abstract

“…Early works in human animation already considered kinematic constraints such as foot contacts [BB98, LCR * XLKvdP20,YTL18] where ground reaction forces are explicitly modelled in the physics engine, and also relates to footplacement strategies that are a real challenge for locomotion policies [PvdP17]. In the following, we overview existing approaches for foot contacts labels detection (2.1) and ground reaction forces estimation (2.2), as well as existing databases of motion data labelled with information on foot contacts (2.3).…”

Section: Related Workmentioning

confidence: 99%

UnderPressure: Deep Learning for Foot Contact Detection, Ground Reaction Force Estimation and Footskate Cleanup

Mourot

Hoyet

Clerc

et al. 2022

Computer Graphics Forum

View full text Add to dashboard Cite

Human motion synthesis and editing are essential to many applications like video games, virtual reality, and film post‐production. However, they often introduce artefacts in motion capture data, which can be detrimental to the perceived realism. In particular, footskating is a frequent and disturbing artefact, which requires knowledge of foot contacts to be cleaned up. Current approaches to obtain foot contact labels rely either on unreliable threshold‐based heuristics or on tedious manual annotation. In this article, we address automatic foot contact label detection from motion capture data with a deep learning based method. To this end, we first publicly release Under Pressure, a novel motion capture database labelled with pressure insoles data serving as reliable knowledge of foot contact with the ground. Then, we design and train a deep neural network to estimate ground reaction forces exerted on the feet from motion data and then derive accurate foot contact labels. The evaluation of our model shows that we significantly outperform heuristic approaches based on height and velocity thresholds and that our approach is much more robust when applied on motion sequences suffering from perturbations like noise or footskate. We further propose a fully automatic workflow for footskate cleanup: foot contact labels are first derived from estimated ground reaction forces. Then, footskate is removed by solving foot constraints through an optimisation‐based inverse kinematics (IK) approach that ensures consistency with the estimated ground reaction forces. Beyond footskate cleanup, both the database and the method we propose could help to improve many approaches based on foot contact labels or ground reaction forces, including inverse dynamics problems like motion reconstruction and learning of deep motion models in motion synthesis or character animation. Our implementation, pre‐trained model as well as links to database can be found at github.com/InterDigitalInc/UnderPressure.

show abstract

“…The output action a t of the control policy at each time step (running at 40Hz) is an 11 dimensional vector with the first 10 entries corresponding to PD targets for the joints, each of which are fed into a PD controller for each joint operating at 2KHz. Prior work has found it advantageous to learn actions in the PD target space rather than directly learning the higherrate actuation commands [15].…”

Section: B Action Spacementioning

confidence: 99%

Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learning

Siekmann

Green

Warila

et al. 2021

Robotics: Science and Systems XVII

122

View full text Add to dashboard Cite

Accurate and precise terrain estimation is a difficult problem for robot locomotion in real-world environments. Thus, it is useful to have systems that do not depend on accurate estimation to the point of fragility. In this paper, we explore the limits of such an approach by investigating the problem of traversing stair-like terrain without any external perception or terrain models on a bipedal robot. For such blind bipedal platforms, the problem appears difficult (even for humans) due to the surprise elevation changes. Our main contribution is to show that sim-to-real reinforcement learning (RL) can achieve robust locomotion over stair-like terrain on the bipedal robot Cassie using only proprioceptive feedback. Importantly, this only requires modifying an existing flat-terrain training RL framework to include stair-like terrain randomization, without any changes in reward function. To our knowledge, this is the first controller for a bipedal, human-scale robot capable of reliably traversing a variety of real-world stairs and other stair-like disturbances using only proprioception.

show abstract

Learning locomotion skills using DeepRL

Cited by 110 publications

References 22 publications

A Survey on Reinforcement Learning Methods in Character Animation

A Survey on Reinforcement Learning Methods in Character Animation

UnderPressure: Deep Learning for Foot Contact Detection, Ground Reaction Force Estimation and Footskate Cleanup

Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learning

Contact Info

Product

Resources

About