Chance-Constrained Trajectory Optimization for Non-linear Systems with Unknown Stochastic Dynamics

Abstract

Iterative trajectory optimization techniques for non-linear dynamical systems are among the most powerful and sample-efficient methods of model-based reinforcement learning and approximate optimal control. By leveraging time-variant local linear-quadratic approximations of system dynamics and reward, such methods can find both a target-optimal trajectory and time-variant optimal feedback controllers. However, the local linear-quadratic assumptions are a major source of optimization bias that leads to catastrophic greedy updates, raising the issue of proper regularization. Moreover, the approximate models' disregard for any physical state-action limits of the system causes further aggravation of the problem, as the optimization moves towards unreachable areas of the stateaction space. In this paper, we address the issue of constrained systems in the scenario of online-fitted stochastic linear dynamics. We propose modeling state and action physical limits as probabilistic chance constraints linear in both state and action and introduce a new trajectory optimization technique that integrates these probabilistic constraints by optimizing a relaxed quadratic program. Our empirical evaluations show a significant improvement in learning robustness, which enables our approach to perform more effective updates and avoid premature convergence observed in state-of-the-art algorithms.

Solving Constrained Trajectory Planning Problems Using Biased Particle Swarm Optimization

“…A comparison is also made between the proposed method and another numerical optimal control solver, named CASADI [34], for solving the constrained atmospheric entry problem (the interior point solver IPOPT [35] is applied in CASADI). This solver has become increasingly popular and it has been applied in the literature to address a number of motion planning or trajectory optimization problems [36], [37]. The four mission cases are solved using CASADI with = 10 −6 as the optimization tolerance.…”

Mediating Between Contact Feasibility and Robustness of Trajectory Optimization Through Chance Complementarity Constraints

“…To trade-off between robustness and constraint satisfaction, chance constraints can be added to an optimization problem to enforce a probabilistic version of the uncertain constraints ( Mesbah, 2016 ; Celik et al, 2019 ; Paulson et al, 2020 ). Chance constraints model uncertainty by defining a probability of constraint satisfaction, which can be tuned to enforce a conservative constraint or to relax the constraint.…”

Sewer orientated framework for ensemble‐based chance‐constrained model predictive control

“…Within the MPC research there are many approaches on how to handle the presence of uncertainty 8‐16 from the robust MPC 16 ; methods aiming for the worst‐case scenario, to the stochastic MPC 8,15 methods aiming for statistic coverage of the uncertainty. There is robust tube‐based MPC (T‐MPC) methods, 16 which utilizing “tubes” to systematical describe all realizations of the uncertainty, such that the control is robust against the worst‐case scenario of the system.…”

“…For this work, we will consider the method known as chance‐constrained (CC) MPC 9‐11,13,16 (CC‐MPC) from the group of stochastic MPC methods as the approach to handling uncertainty. The CC‐MPC methods utilizes knowledge about the stochastic distribution of the uncertainties, to formulate each of the inequality constraints as probabilistic constraints with a chosen probability of being true.…”

“…The past research of CC‐MPC include focusing on ensuring feasibility, 13,16 simplifying the formulation of fx. joint probabilistic constraints, 9,10 and applicability to the case of sewer systems 11…”