A Differentiable Physics Engine for Deep Learning in Robotics

Abstract

An important field in robotics is the optimization of controllers. Currently, robots are often treated as a black box in this optimization process, which is the reason why derivative-free optimization methods such as evolutionary algorithms or reinforcement learning are omnipresent. When gradient-based methods are used, models are kept small or rely on finite difference approximations for the Jacobian. This method quickly grows expensive with increasing numbers of parameters, such as found in deep learning. We propose the implementation of a modern physics engine, which can differentiate control parameters. This engine is implemented for both CPU and GPU. Firstly, this paper shows how such an engine speeds up the optimization process, even for small problems. Furthermore, it explains why this is an alternative approach to deep Q-learning, for using deep learning in robotics. Finally, we argue that this is a big step for deep learning in robotics, as it opens up new possibilities to optimize robots, both in hardware and software.

ChainQueen: A Real-Time Differentiable Physical Simulator for Soft Robotics

“…For rigid bodies, [22], [3] and [4] proposed to approximate object interaction with neural nets; later, [23] explored their usage in control. Approximate analytic differentiable rigid body simulators have also been proposed [5], [24]. Such systems have been deployed for manipulation and planning [25].…”

“…In particular, differentiable physical simulators enable the use of gradient-based optimizers, significantly improving control efficiency and precision. Motivated by this, there has been extensive research on differentiable rigid body simulators, using approximate [3], [4] and exact [5], [6], [7] methods.…”

Propagation Networks for Model-Based Control Under Partial Observation

“…In recent years, researchers have been building differentiable physics simulators in various forms [1], [3], [13], [14]. Our (a) shows the initial states of a Newton's cradle, based on which both the Interaction Networks and Propagation Networks try to predict future states; (b-i) The Interaction Networks can only propagate the force along a single relation at a time step, thus results in a false prediction (c-i); (b-ii) Our proposed method can propagate the force correctly which leads to the correct prediction (c-ii); (d) A downstream task where we aim to achieve a specific goal using the learned model; (e-i) Model-based control methods fail to produce the correct control using Interaction Networks while (e-ii) our model can provide the desired control signal.…”

“…For example, approximate, analytical differentiable rigid body simulators [14], [15] have been deployed for tool manipulation and tool-use planning [16].…”

Combining Physical Simulators and Object-Based Networks for Control

“…There has been an increasing interest in building differentiable physics simulators [18]. For example, Degrave et al [19] proposed to directly solve differentiable equations. Such systems have been deployed for manipulation and planning for tool use [20].…”