Rethinking Optimization with Differentiable Simulation from a Global Perspective

Abstract

Differentiable simulation is a promising toolkit for fast gradient-based policy optimization and system identification. However, existing approaches to differentiable simulation have largely tackled scenarios where obtaining smooth gradients has been relatively easy, such as systems with mostly smooth dynamics. In this work, we study the challenges that differentiable simulation presents when it is not feasible to expect that a single descent reaches a global optimum, which is often a problem in contact-rich scenarios. We analyze the optimization landscapes of diverse scenarios that contain both rigid bodies and deformable objects. In dynamic environments with highly deformable objects and fluids, differentiable simulators produce rugged landscapes with nonetheless useful gradients in some parts of the space. We propose a method that combines Bayesian optimization with semi-local 'leaps' to obtain a global search method that can use gradients effectively, while also maintaining robust performance in regions with noisy gradients. We show that our approach outperforms several gradient-based and gradient-free baselines on an extensive set of experiments in simulation, and also validate the method using experiments with a real robot and deformables. Videos and supplementary materials are available at https://tinyurl.com/globdiff.

GOATS: Goal Sampling Adaptation for Scooping with Curriculum Reinforcement Learning

“…Wang et al learn generative models for scoop manipulation skills [2]. Antonova et al employ Bayesian Optimization with differentiable simulation to scoop a cube from the water in a tank and raise it to the highest possible position [3]. ToolFlowNet uses behavior cloning from point clouds to perform water pouring and ball scooping tasks [4].…”

“…This skill can be generalized to a variety of tasks, from ladling soup and collecting peas with a spoon on a dining table to excavating soils on a construction site. Despite numerous studies in the field of robotic automation, only a small amount of works have studied autonomous scooping for robots, such as [1], [2], [3], [4]. Scooping is still a challenging task due to the highdimensional interaction space between the end-effector and the dynamic materials for scooping.…”

“…In this work, we formulate the problem of goalconditioned water scooping, and propose a goal sampling 1 Carnegie Mellon University 2 Robotics and Autonomous Driving Lab, Baidu Research, USA 3 The University of Hong Kong * Work done during the internship at Baidu Research †Authors contributed equally to this work Fig. 1: This figure depicts our goal-conditioned water scooping tasks.…”

FluidLab: A Differentiable Environment for Benchmarking Complex Fluid Manipulation

“…In recent years, there has been a rising interest in manipulation problems concerning systems with deformable objects, including elastic objects such as cloth (Liang et al, 2019;Wu et al, 2019;Lin et al, 2022b), as well as plastic objects like plasticine (Huang et al, 2021). Some of them also study manipulation problems involving fluids, but mainly consider relatively simple problems settings such as robotic pouring (Sieb et al, 2020;Narasimhan et al, 2022) or scooping (Antonova et al, 2022), and mostly deal with only water (Schenck & Fox, 2018;Ma et al, 2018;Seita et al, 2022;Antonova et al, 2022). In this work, we aim to study a comprehensive set of manipulation tasks involving different fluid types with various properties.…”

“…Prior works in robotic manipulation covering fluids mostly adopt relatively simple task settings, and usually consider tasks with a single-phase fluid, such as pouring water (Schenck & Fox, 2018;Sermanet et al, 2018) or scooping objects from water (Seita et al, 2022;Antonova et al, 2022). In this work, we aim to study the problem of complex fluid manipulation with more challenging task settings and complex goal configurations.…”