Squash-Box Feasibility Driven Differential Dynamic Programming

Abstract

Recently, Differential Dynamic Programming (DDP) and other similar algorithms have become the solvers of choice when performing non-linear Model Predictive Control (nMPC) with modern robotic devices. The reason is that they have a lower computational cost per iteration when compared with off-the-shelf Non-Linear Programming (NLP) solvers, which enables its online operation. However, they cannot handle constraints, and are known to have poor convergence capabilities. In this paper, we propose a method to solve the optimal control problem with control bounds through a squashing function (i.e., a sigmoid, which is bounded by construction). It has been shown that a naive use of squashing functions damage the convergence rate. To tackle this, we first propose to add a quadratic barrier that avoids the difficulty of the plateau produced by the sigmoid. Second, we add an outer loop that adapts both the sigmoid and the barrier; it makes the optimal control problem with the squashing function converge to the original control-bounded problem. To validate our method, we present simulation results for different types of platforms including a multi-rotor, a biped, a quadruped and a humanoid robot.

Trajectory Optimization with Optimization-Based Dynamics

“…Box constraints have been incorporated at each step in the backward pass in order to enforce control limits [32]. An alternative approach embeds controls in barrier functions which smoothly approximate constraints [19]. Augmented Lagrangian methods and constrained backward and forward passes have also been proposed for handling general constraints [9,40,13].…”

“…Relaxing the complementarity constraint via a central-path parameter, introducing a slack variable for the signed-distance function φ, and combining this reformulation with the system's dynamics results in a problem formulation (12)(13)(14)(15) that can be optimized with Algorithm 1. Unlike approaches that include joint limits at the solver level, including the impact (19) and limit (18) constraints at the dynamics level enables impact forces encountered at joint stops to be optimized and applied to the system.…”

Optimizing Trajectories with Closed-Loop Dynamic SQP

“…iLQR/DDP-like methods for constrained trajectory optimization fall into one of three main categories: control-bounds only [14], [15], modification of the backward pass via KKT analysis [16]- [20], and augmented Lagrangian methods [3], [4], [21]- [23]. In the first category, only control-bound constraints are considered.…”

“…For instance, [14] leverage the box-constrained DDP algorithm where the projected Newton algorithm is used to compute the affine perturbation policy in the backward pass, accounting for the box constraints on the control input. On the other hand, [15] composes a "squashing" function to constrain the input, and augments the objective with a barrier penalty to encourage the iterates to stay away from the plateaued regions of squashing function.…”

“…Related Work: Quasi-DDP methods for constrained trajectory optimization fall into one of three main categories: control-bounds only [14], [15], modified backward pass via KKT analysis [16]- [20], and augmented Lagrangian methods [3], [4], [21]- [23]. We provide a comprehensive overview of these approaches in Appendix A.…”