Sample-Based Bounds for Coherent Risk Measures: Applications to Policy Synthesis and Verification

Abstract

The dramatic increase of autonomous systems subject to variable environments has given rise to the pressing need to consider risk in both the synthesis and verification of policies for these systems. This paper aims to address a few problems regarding risk-aware verification and policy synthesis, by first developing a sample-based method to bound the risk measure evaluation of a random variable whose distribution is unknown. These bounds permit us to generate high-confidence verification statements for a large class of robotic systems. Second, we develop a sample-based method to determine solutions to non-convex optimization problems that outperform a large fraction of the decision space of possible solutions. Both sample-based approaches then permit us to rapidly synthesize risk-aware policies that are guaranteed to achieve a minimum level of system performance. To showcase our approach in simulation, we verify a cooperative multi-agent system and develop a risk-aware controller that outperforms the system's baseline controller. We also mention how our approach can be extended to account for any g-entropic risk measure -the subset of coherent risk measures on which we focus.

Safe Reinforcement Learning with Probabilistic Guarantees Satisfying Temporal Logic Specifications in Continuous Action Spaces

“…For a more detailed derivation of this theorem and its implications on dimensional scaling, we refer the interested reader to [13].…”

“…To provide probabilistic safety guarantees, we leverage existing work in the probabilistic verification literature taking a scenario approach to risk-aware probabilistic verification [13], [14]. Here, the standard approach as described in [15] is to pose verification as an optimization problem minimizing a quantifiable satisfaction measure provided by either a temporal logic specification or another method.…”