Learning a unified control policy for safe falling

Visak, Kumar,; Ha, Sehoon; Liu, C. Karen

doi:10.1109/iros.2017.8206246

Cited by 17 publications

(10 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Non‐locomotion tasks are also important for full‐body character animation. Kumar et al [KHL17] use an algorithm based on MACE for the task of teaching a virtual humanoid model to safely fall by minimizing the maximal impulse experienced by its body. They train a mixture of actor‐critic networks associated with all possible contacting body parts, and further use a form of hierarchical reinforcement learning, with an abstract policy deciding the high‐level behavior, and a joint policy responsible for actually executing the action.…”

Section: Skeletal Animationmentioning

confidence: 99%

A Survey on Reinforcement Learning Methods in Character Animation

Kwiatkowski

Alvarado

Kalogeiton

et al. 2022

Computer Graphics Forum

Self Cite

View full text Add to dashboard Cite

Reinforcement Learning is an area of Machine Learning focused on how agents can be trained to make sequential decisions, and achieve a particular goal within an arbitrary environment. While learning, they repeatedly take actions based on their observation of the environment, and receive appropriate rewards which define the objective. This experience is then used to progressively improve the policy controlling the agent's behavior, typically represented by a neural network. This trained module can then be reused for similar problems, which makes this approach promising for the animation of autonomous, yet reactive characters in simulators, video games or virtual reality environments. This paper surveys the modern Deep Reinforcement Learning methods and discusses their possible applications in Character Animation, from skeletal control of a single, physically‐based character to navigation controllers for individual agents and virtual crowds. It also describes the practical side of training DRL systems, comparing the different frameworks available to build such agents.

show abstract

Section: Skeletal Animationmentioning

confidence: 99%

A Survey on Reinforcement Learning Methods in Character Animation

Kwiatkowski

Alvarado

Kalogeiton

et al. 2022

Computer Graphics Forum

Self Cite

View full text Add to dashboard Cite

show abstract

“…In addition to enforcing general safety constraints, researchers have also investigated more task-dependent safety problems. For example, for humanoid robots, researchers have devised specialized algorithms to reduce the damage they receives during falling [28]- [31]. Though effective in handling the falling scenario, these methods may not generalize to other types of failure modes.…”

Section: Related Workmentioning

confidence: 99%

Protective Policy Transfer

Yu¹,

Liu²,

Turk³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Being able to transfer existing skills to new situations is a key capability when training robots to operate in unpredictable real-world environments. A successful transfer algorithm should not only minimize the number of samples that the robot needs to collect in the new environment, but also prevent the robot from damaging itself or the surrounding environment during the transfer process. In this work, we introduce a policy transfer algorithm for adapting robot motor skills to novel scenarios while minimizing serious failures. Our algorithm trains two control policies in the training environment: a task policy that is optimized to complete the task of interest, and a protective policy that is dedicated to keep the robot from unsafe events (e.g. falling to the ground). To decide which policy to use during execution, we learn a safety estimator model in the training environment that estimates a continuous safety level of the robot. When used with a set of thresholds, the safety estimator becomes a classifier for switching between the protective policy and the task policy. We evaluate our approach on four simulated robot locomotion problems and a 2D navigation problem and show that our method can achieve successful transfer to notably different environments while taking the robot's safety into consideration.

show abstract

“…In [19] the authors presented an optimization-based control strategy to generate whole-body trajectories to minimize fall damage. Given an unstable initial state of the robot, Liu et al [20], [21] found the optimal contact sequence to dissipate the initial momentum with minimal impacts on the robot. These fall-damage minimization algorithms could be used in combination with our algorithm, in case a fall is predicted and balance seems impossible to recover.…”

Section: B State Of the Artmentioning

confidence: 99%

Zero Step Capturability for Legged Robots in Multicontact

2018

View full text Add to dashboard Cite

The ability to anticipate a fall is fundamental for any robot that has to balance. Currently, fast fall-prediction algorithms only exist for simple models, such as the Linear Inverted Pendulum Model (LIPM), whose validity breaks down in multi-contact scenarios (i.e. when contacts are not limited to a flat ground). This paper presents a fast fall-prediction algorithm based on the point-mass model, which remains valid in multicontact scenarios. The key assumption of our algorithm is that, in order to come to a stop without changing its contacts, a robot only needs to accelerate its center of mass in the direction opposite to its velocity. This assumption allows us to predict the fall by means of a convex optimal control problem, which we solve with a fast custom algorithm (less than 11 ms of computation time). We validated the approach through extensive simulations with the humanoid robot HRP-2 in randomly-sampled scenarios. Comparisons with standard LIPM-based methods demonstrate the superiority of our algorithm in predicting the fall of the robot, when controlled with a state-of-the-art balance controller. This work lays the foundations for the solution of the challenging problem of push recovery in multi-contact scenarios.

show abstract

Learning a unified control policy for safe falling

Cited by 17 publications

References 14 publications

A Survey on Reinforcement Learning Methods in Character Animation

A Survey on Reinforcement Learning Methods in Character Animation

Protective Policy Transfer

Zero Step Capturability for Legged Robots in Multicontact

Contact Info

Product

Resources

About