Differentiable Physics Models for Real-world Offline Model-based Reinforcement Learning

Lutter, Michael; Silberbauer, Johannes; Watson, Joe; Peters, Jan

doi:10.1109/icra48506.2021.9561805

Cited by 26 publications

(14 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, it was quickly observed that the inferred parameters may be physically implausible, leading to the development of methods that can account for this ( Ting et al, 2011 ). With the advent of deep learning, such structured physics-based approaches have been enhanced with NNs, yielding nonlinear system identification methods such as the ones based on the Newton-Euler forward dynamics ( Sutanto et al, 2020 ; Lutter et al, 2021b ). Alternatively, the simulator can be augmented with a NN to learn the domain parameter residuals, minimizing the one step prediction error ( Allevato et al, 2019 ).…”

Section: Relation Of Sim-to-real To Other Fieldsmentioning

confidence: 99%

Robot Learning From Randomized Simulations: A Review

et al. 2022

Self Cite

View full text Add to dashboard Cite

The rise of deep learning has caused a paradigm shift in robotics research, favoring methods that require large amounts of data. Unfortunately, it is prohibitively expensive to generate such data sets on a physical platform. Therefore, state-of-the-art approaches learn in simulation where data generation is fast as well as inexpensive and subsequently transfer the knowledge to the real robot (sim-to-real). Despite becoming increasingly realistic, all simulators are by construction based on models, hence inevitably imperfect. This raises the question of how simulators can be modified to facilitate learning robot control policies and overcome the mismatch between simulation and reality, often called the “reality gap.” We provide a comprehensive review of sim-to-real research for robotics, focusing on a technique named “domain randomization” which is a method for learning from randomized simulations.

show abstract

Section: Relation Of Sim-to-real To Other Fieldsmentioning

confidence: 99%

Robot Learning From Randomized Simulations: A Review

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…This hybrid simulator shows the ability to learn complex dynamics from real data by replacing the Quadratic programming (QP) solver inside a model-predictive controller with neural layers. Some other differentiable rigid simulated are introduced in [22], [23], [24]. However, they are mostly applied to learning-based tasks with adaptation to neural networks.…”

Section: A Differentiable Rigid Physical Simulationmentioning

confidence: 99%

“…For joint-space values, we can assume [p, ṗ, p] are the joint level position, velocity, and acceleration respectively, and τ are torques applied to the joints. Werling and Lutter [10], [50] present the Recursive-Newton-Euler (RNE) algorithm for the kinetic and potential models for differentiability.…”

Section: A Parameter Identificationmentioning

confidence: 99%

Parameter Identification and Motion Control for Articulated Rigid Body Robots Using Differentiable Position-based Dynamics

Liu¹,

Li²,

Lu³

et al. 2022

Preprint

View full text Add to dashboard Cite

“…Structured Policies Across machine learning, inductive biases are means of introducing domain knowledge to improve sample efficiency, interpretability and reliability. In the context of control, inductive biases have been applied to enhance models for model-based RL [21], or to policies to simplify or improve policy search. Popular structures include options [22], dynamic movement primitives [23,24,25], autoregressive models [26], model predictive control [27] and motion planners [28,29].…”

Section: Related Workmentioning

confidence: 99%

Benchmarking Structured Policies and Policy Optimization for Real-World Dexterous Object Manipulation

Funk,

Schaff,

Madan

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Dexterous manipulation is a challenging and important problem in robotics. While data-driven methods are a promising approach, current benchmarks require simulation or extensive engineering support due to the sample inefficiency of popular methods. We present benchmarks for the TriFinger system, an open-source robotic platform for dexterous manipulation and the focus of the 2020 Real Robot Challenge. The benchmarked methods, which were successful in the challenge, can be generally described as structured policies, as they combine elements of classical robotics and modern policy optimization. This inclusion of inductive biases facilitates sample efficiency, interpretability, reliability and high performance. The key aspects of this benchmarking is validation of the baselines across both simulation and the real system, thorough ablation study over the core features of each solution, and a retrospective analysis of the challenge as a manipulation benchmark. The code and demo videos for this work can be found on our website (https://sites.google.com/view/benchmark-rrc).

show abstract

Differentiable Physics Models for Real-world Offline Model-based Reinforcement Learning

Cited by 26 publications

References 23 publications

Robot Learning From Randomized Simulations: A Review

Robot Learning From Randomized Simulations: A Review

Parameter Identification and Motion Control for Articulated Rigid Body Robots Using Differentiable Position-based Dynamics

Benchmarking Structured Policies and Policy Optimization for Real-World Dexterous Object Manipulation

Contact Info

Product

Resources

About