Bundled Gradients Through Contact Via Randomized Smoothing

Terry, Suh, H. J.; Pang, Tao; Tedrake, Russ

doi:10.1109/lra.2022.3146931

Cited by 24 publications

(15 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To enforce the convergence towards an optimal (local) solution, it remains crucial to reduce the noise injected via the randomized smoothing across the iterations. A first possible strategy [13] consists in relying on Robbins-Monro rule [42] by decreasing the variance in a way such that k 2 k < ∞ to guarantee convergence towards a local minima. In this work, we propose to decrease k in a way that adapts to the problem and avoids the smoothing being reduced too quickly, which would lead to performance similar to classical DDP, or too slowly, which would induce an unnecessary large number of iterations.…”

Section: B Adaptive Smoothingmentioning

confidence: 99%

See 1 more Smart Citation

Leveraging Randomized Smoothing for Optimal Control of Nonsmooth Dynamical Systems

Lidec

Louis

Schmid

et al. 2022

Preprint

View full text Add to dashboard Cite

Optimal control (OC) algorithms such as Differential Dynamic Programming (DDP) take advantage of the derivatives of the dynamics to efficiently control physical systems. Yet, in the presence of nonsmooth dynamical systems, such class of algorithms are likely to fail due, for instance, to the presence of discontinuities in the dynamics derivatives or because of non-informative gradient. On the contrary, reinforcement learning (RL) algorithms have shown better empirical results in scenarios exhibiting non-smooth effects (contacts, frictions, etc). Our approach leverages recent works on randomized smoothing (RS) to tackle non-smoothness issues commonly encountered in optimal control, and provides key insights on the interplay between RL and OC through the prism of RS methods. This naturally leads us to introduce the randomized Differential Dynamic Programming (R-DDP) algorithm accounting for deterministic but non-smooth dynamics in a very sample-efficient way. The experiments demonstrate that our method is able to solve classic robotic problems with dry friction and frictional contacts, where classical OC algorithms are likely to fail and RL algorithms require in practice a prohibitive number of samples to find an optimal solution.

show abstract

Section: B Adaptive Smoothingmentioning

confidence: 99%

“…Concurrently to our work, [13] introduced the notion of randomized smoothing in order to get gradients through contacts. We take a different point of view by establishing links between RL and OC through the lens of RS, which is a first step towards a stronger interplay between these two fields.…”

Section: Introductionmentioning

confidence: 99%

Leveraging Randomized Smoothing for Optimal Control of Nonsmooth Dynamical Systems

Lidec

Louis

Schmid

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Although it is possible to solve nonlinear optimization problems without access to the gradients of the objective or constraint functions, either by estimating gradients [16] or using zero-order methods [17], it is often much faster to use exact gradient information when it is available. However, exact gradients can be difficult to derive symbolically for complex optimization problems.…”

Section: A Differentiable Simulation and Temporal Logicmentioning

confidence: 99%

Robust Counterexample-guided Optimization for Planning from Differentiable Temporal Logic

Dawson¹,

Fan²

2022

Preprint

View full text Add to dashboard Cite

Signal temporal logic (STL) provides a powerful, flexible framework for specifying complex autonomy tasks; however, existing methods for planning based on STL specifications have difficulty scaling to long-horizon tasks and are not robust to external disturbances. In this paper, we present an algorithm for finding robust plans that satisfy STL specifications. Our method alternates between local optimization and local falsification, using automatically differentiable temporal logic to iteratively optimize its plan in response to counterexamples found during the falsification process. We benchmark our counterexample-guided planning method against state-of-the-art planning methods on two long-horizon satellite rendezvous missions, showing that our method finds high-quality plans that satisfy STL specifications despite adversarial disturbances. We find that our method consistently finds plans that are robust to adversarial disturbances and requires less than half the time of competing methods. We provide an implementation of our planner at https://github.com/MIT-REALM/architect.

show abstract

“…On the other hand, AD necessarily incurs some overhead at runtime, making each AD function call more expensive than those used in an FD scheme. Additionally, some arguments [17] suggest that exact gradients may be less useful than finitedifference or stochastic approximations when the objective is stiff or discontinuous. We compare AD with a 3-point finitedifference method by re-solving problem (2) for both case studies, keeping all parameters constant (N = 512, λ = 0.1, same random seed) and substituting the gradients obtained using AD for those computed using finite differences.…”

Section: Design Optimization Ablation Studymentioning

confidence: 99%

“…In recent years, the robotics community has also developed special-purpose differentiable simulators for robotic systems, particularly those involving rigid body contact dynamics [14,15,16,17]. These simulators have been used to solve system identification and controller design tasks, but they do not represent a general-purpose framework, as gradients are often derived by hand and the simulators are not expressive enough to model full-stack robotic systems (e.g.…”

Section: Introductionmentioning

confidence: 99%

Certifiable Robot Design Optimization using Differentiable Programming

Dawson¹,

Fan²

2022

Preprint

View full text Add to dashboard Cite

There is a growing need for computational tools to automatically design and verify autonomous systems, especially complex robotic systems involving perception, planning, control, and hardware in the autonomy stack. Differentiable programming has recently emerged as powerful tool for modeling and optimization. However, very few studies have been done to understand how differentiable programming can be used for robust, certifiable end-to-end design optimization. In this paper, we fill this gap by combining differentiable programming for robot design optimization with a novel statistical framework for certifying the robustness of optimized designs. Our framework can conduct end-to-end optimization and robustness certification for robotics systems, enabling simultaneous optimization of navigation, perception, planning, control, and hardware subsystems.Using simulation and hardware experiments, we show how our tool can be used to solve practical problems in robotics. First, we optimize sensor placements for robot navigation (a design with 5 subsystems and 6 tunable parameters) in under 5 minutes to achieve an 8.4x performance improvement compared to the initial design. Second, we solve a multi-agent collaborative manipulation task (3 subsystems and 454 parameters) in under an hour to achieve a 44% performance improvement over the initial design. We find that differentiable programming enables much faster (32% and 20x, respectively for each example) optimization than approximate gradient methods. We certify the robustness of each design and successfully deploy the optimized designs in hardware. An open-source implementation is available at https://github.com/ MIT-REALM/architect.

show abstract

Bundled Gradients Through Contact Via Randomized Smoothing

Cited by 24 publications

References 45 publications

Leveraging Randomized Smoothing for Optimal Control of Nonsmooth Dynamical Systems

Leveraging Randomized Smoothing for Optimal Control of Nonsmooth Dynamical Systems

Robust Counterexample-guided Optimization for Planning from Differentiable Temporal Logic

Certifiable Robot Design Optimization using Differentiable Programming

Contact Info

Product

Resources

About