Risk-averse autonomous systems: A brief history and recent developments from the perspective of optimal control

Abstract

We offer a historical overview of methodologies for quantifying the notion of risk and optimizing risk-aware autonomous systems, with emphasis on riskaverse settings in which safety may be critical. We categorize and present stateof-the-art approaches, and we describe connections between such approaches and ideas from the fields of decision theory, operations research, reinforcement learning, and stochastic control. The first part of the review focuses on model-based risk-averse methods. The second part discusses methods that blend model-based and model-free techniques for the purpose of designing policies with improved adaptive capabilities. We conclude by highlighting areas for future research.

RAP: Risk-Aware Prediction for Robust Planning

“…More recent methods include risk-sensitive nonlinear MPC [6,52], Q-learning [44,53], and actor-critic [54,55] methods, for various types of risk measures. Refer to a recent survey [56] for further details. Unlike those methods in which the policy directly optimizes a risk-measure, we propose to instead bias the prediction so that risk-sensitivity can be achieved by a risk-neutral planner that simply optimizes the expected value of the cost.…”