A Framework for Time-Consistent, Risk-Sensitive Model Predictive Control: Theory and Algorithms

Abstract

In this paper we present a framework for risk-sensitive model predictive control (MPC) of linear systems affected by stochastic multiplicative uncertainty. Our key innovation is to consider a time-consistent, dynamic risk evaluation of the cumulative cost as the objective function to be minimized. This framework is axiomatically justified in terms of time-consistency of risk assessments, is amenable to dynamic optimization, and is unifying in the sense that it captures a full range of risk preferences from risk-neutral (i.e., expectation) to worst case. Within this framework, we propose and analyze an online risk-sensitive MPC algorithm that is provably stabilizing. Furthermore, by exploiting the dual representation of time-consistent, dynamic risk measures, we cast the computation of the MPC control law as a convex optimization problem amenable to real-time implementation. Simulation results are presented and discussed.

STEP: Stochastic Traversability Evaluation and Planning for Risk-Aware Off-road Navigation

“…In recent years, Ahmadi et al synthesized risk averse optimal policies for partially observable MDPs, constrained MDPs, and for shortest path problems in MDPs [4,3,5]. Coherent risk measures have been used in a MPC framework when the system model is uncertain [36] and when the uncertainty is a result of measurement noise or moving obstacles [12]. In [20,12], the authors incorporated risk constraints in the form of distance to the randomly moving obstacles but did not include model uncertainty.…”

Value at Risk strategies for robot swarms in hazardous environments

“…In the field of robotics, the relevance of VaR as a risk metric has recently been noted in the area of motion planning and control, in the form of conditional value-at-risk (CVaR). [11][12][13][14][15] CVaR, also known as expected shortfall (ES), is a more conservative measure than VaR which better accounts for tail risk (rare but dangerous events). The CVaR of a random loss represents the conditional expectation of the loss within the (1 − α) worstcase quantile of the loss distribution, where α ∈ (0, 1).…”

“…16 For a random variable X, CVaR and VaR are related as follows: CVaR α (X) = E[X|X VaR α (X)]. CVaR has been preferred in recent research [11][12][13][14][15] because it is noted to be a 'coherent' risk measure, in respect of certain formal characteristics, 9 which are arguably important for risk assessment rationality. 12 Practically speaking, because CVaR gives more conservative risk estimates, here we generally use VaR for our risk metric because we investigate deployment of 'expendable' robot swarms where some amount of tail risk is tolerated by the user.…”

Risk-Averse Control via CVaR Barrier Functions: Application to Bipedal Robot Locomotion

“…An important example of a coherent risk measure is the conditional value-at-risk (CVaR) that has received significant attention in decision making problems, such as Markov decision processes (MDPs) [6], [7], [8]. For stochastic discretetime dynamical systems, a model predictive control technique with coherent risk objectives was proposed in [9], wherein the authors also proposed Lyapunov conditions for risk-sensitive exponential stability. Moreover, a method based on stochastic reachability analysis was proposed in [10] to estimate a CVaRsafe set of initial conditions via the solution to an MDP.…”