Safe Multi-objective Planning with a Posteriori Preferences

Abstract

Autonomous planning in safety critical systems is a difficult task where decisions must carefully balance optimisation for performance goals of the system while also keeping the system away from safety hazards. These tasks often conflict, and hence present a challenging multi-objective planning problem where at least one of the objectives relates to safety risk. Recasting safety risk into an objective introduces additional requirements on planning algorithms: safety risk cannot be "averaged out" nor can it be combined with other objectives without loss of information and losing its intended purpose as a tool in risk reduction. Thus, existing algorithms for multi-objective planning cannot be used directly as they do not provide any facility to accurately track and update safety risk. A common workaround is to restrict available decisions to those guaranteed safe a priori, but this can be overly conservative and hamper performance significantly. In this paper, we propose a planning algorithm based on multiobjective Monte-Carlo Tree Search to resolve these problems by recognising safety risk as a first class objective. Our algorithm explicitly models the safety of the system separately from the performance of the system, uses safety risk to both optimise and provide constraints for safety in the planning process, and uses an ALARP-based preference selection method to choose an appropriate safe plan from its output. The preference selection method chooses from the set of multiple safe plans to weigh risk against performance. We demonstrate the behaviour of the algorithm using an example representative of safety critical decision-making.

Multi-cost Bounded Reachability in MDP

“…This makes many planning problems inherently multi-objective [12,17]. In particular, safety properties cannot be averaged out by good performance [21]. Planning scenarios in various application areas [44] have different resource constraints.…”

NASA Formal Methods

“…Eastwood et al [3] present an algorithm for finding permissive robot action plans optimal w.r.t. to safety and performance.…”

“…A risk structure is a weighted labeled transition system pΣ, A, ∆, Wq with a set Σ called the risk state space (Definition 2), a set A of actions used as transition labels, a relation ∆ Ď Σ ˆA ˆΣ called labeled transition relation, and a set W of partial functions w : pΣ Y A Y ∆q Ñ W w called weights where the set W w can be, e.g. N, R, r0, 1s, or tm, c, f u 3.…”

Multi-cost Bounded Tradeoff Analysis in MDP

“…This tradeoff makes many planning problems inherently multi-objective [12,17]. In particular, safety properties cannot be averaged out by good performance [22]. Planning scenarios in various application areas [51] have different resource constraints.…”