A Survey on Reinforcement Learning Methods in Character Animation

Kwiatkowski, Ariel; Alvarado, Eduardo; Kalogeiton, Vicky; Liu, C. Karen; Pettré, Julien; Panne, Michiel van de; Cani, Marie-Paule

doi:10.1111/cgf.14504

Cited by 18 publications

(10 citation statements)

References 117 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Control policies have also been learned which are conditioned on not only the desired motion, but also the specific morphology of a simulated character, which can then even be changed at run time [Won and Lee 2019]. We further refer the reader to a recent survey of RL-related animation methods [Kwiatkowski et al 2022]. We build on the foundations provided above for our specific problem, namely how to retarget from sparse (and therefore potentially highly ambiguous) input data to a non-human physics-based character with very different dimensions and proportions.…”

Section: Physics-based Character Simulationmentioning

confidence: 99%

Physics-based Motion Retargeting from Sparse Inputs

Reda,

Won,

et al. 2023

Proc. ACM Comput. Graph. Interact. Tech.

View full text Add to dashboard Cite

Avatars are important to create interactive and immersive experiences in virtual worlds. One challenge in animating these characters to mimic a user's motion is that commercial AR/VR products consist only of a headset and controllers, providing very limited sensor data of the user's pose. Another challenge is that an avatar might have a different skeleton structure than a human and the mapping between them is unclear. In this work we address both of these challenges. We introduce a method to retarget motions in real-time from sparse human sensor data to characters of various morphologies. Our method uses reinforcement learning to train a policy to control characters in a physics simulator. We only require human motion capture data for training, without relying on artist-generated animations for each avatar. This allows us to use large motion capture datasets to train general policies that can track unseen users from real and sparse data in real-time. We demonstrate the feasibility of our approach on three characters with different skeleton structure: a dinosaur, a mouse-like creature and a human. We show that the avatar poses often match the user surprisingly well, despite having no sensor information of the lower body available. We discuss and ablate the important components in our framework, specifically the kinematic retargeting step, the imitation, contact and action reward as well as our asymmetric actor-critic observations. We further explore the robustness of our method in a variety of settings including unbalancing, dancing and sports motions.

show abstract

Section: Physics-based Character Simulationmentioning

confidence: 99%

Physics-based Motion Retargeting from Sparse Inputs

Reda,

Won,

et al. 2023

Proc. ACM Comput. Graph. Interact. Tech.

View full text Add to dashboard Cite

show abstract

“…However, the physical model needs to be carefully designed, as it may heavily impact the learned policies. 19 Some DRL methods learn more natural physically actuated motion with a reward function that restricts character effort and energy, and maximizes symmetry and stability. [20][21][22] DeepLoco 23 can also perform collision-free locomotion with the aid of a height map of the obstacles of the scene with a two-level approach, where a higher-level controller is later refined with a lower-level controller, but only for environments with a single agent.…”

Section: Related Workmentioning

confidence: 99%

“…Physics‐based controllers address some of these problems by performing learning inside a world with physics. However, the physical model needs to be carefully designed, as it may heavily impact the learned policies 19 . Some DRL methods learn more natural physically actuated motion with a reward function that restricts character effort and energy, and maximizes symmetry and stability 20‐22 .…”

Section: Related Workmentioning

confidence: 99%

Skeleton‐level control for multi‐agent simulation through deep reinforcement learning

Alonso

Jin

2022

Computer Animation & Virtual

View full text Add to dashboard Cite

Multi-agent simulation has attracted much attention in the field of computer animation in the last decades for its ability to model interaction between autonomous micro level entities. It widely uses deep reinforcement learning (DRL), which allows us to model environments and its agents approaching real-world and human-level complexity, with applications in robotics and computer animation, among others. However, DRL multi-agent simulation faces additional challenges: they have to be able to generalize high-dimensional observations and relate them with a high-dimensional action space, maximizing long-term cumulated reward. Due to this, DRL systems with numerous interacting agents seldom consider skeleton-level action spaces. To this end, we present skeleton-level control for multi-agent simulation with DRL. With our method, we are able to procedurally generate real-time collision-free simulations directly on individual agents with a high-dimensional skeleton-level action space. The state in our DRL system includes the velocity of the agent, its destination, and the status of its joints, as well as visual-based information about the environment and other agents. Our reward function encourages motion into the target destinations, and enalizes collision. We provide extensive experimentation to show the ability of agents to reach their goal through its skeleton motion while successfully avoiding inter-collisions.

show abstract

“…The use of Reinforcement Learning (RL) has become prevalent in recent years for optimizing physically‐based controller parameters [KAK*22]. These approaches can preserve character balance to achieve realistic locomotion up to complex acrobatic gestures [CMM*18,PALvdP18,PCZ*20].…”

Section: Related Workmentioning

confidence: 99%

Generating Upper‐Body Motion for Real‐Time Characters Making their Way through Dynamic Environments

Alvarado

Rohmer

Cani

2022

Computer Graphics Forum

Self Cite

View full text Add to dashboard Cite

Real‐time character animation in dynamic environments requires the generation of plausible upper‐body movements regardless of the nature of the environment, including non‐rigid obstacles such as vegetation. We propose a flexible model for upper‐body interactions, based on the anticipation of the character's surroundings, and on antagonistic controllers to adapt the amount of muscular stiffness and response time to better deal with obstacles. Our solution relies on a hybrid method for character animation that couples a keyframe sequence with kinematic constraints and lightweight physics. The dynamic response of the character's upper‐limbs leverages antagonistic controllers, allowing us to tune tension/relaxation in the upper‐body without diverging from the reference keyframe motion. A new sight model, controlled by procedural rules, enables high‐level authoring of the way the character generates interactions by adapting its stiffness and reaction time. As results show, our real‐time method offers precise and explicit control over the character's behavior and style, while seamlessly adapting to new situations. Our model is therefore well suited for gaming applications.

show abstract

A Survey on Reinforcement Learning Methods in Character Animation

Cited by 18 publications

References 117 publications

Physics-based Motion Retargeting from Sparse Inputs

Physics-based Motion Retargeting from Sparse Inputs

Skeleton‐level control for multi‐agent simulation through deep reinforcement learning

Generating Upper‐Body Motion for Real‐Time Characters Making their Way through Dynamic Environments

Contact Info

Product

Resources

About