Towards the Verification of Safety-critical Autonomous Systems in Dynamic Environments

Abstract

There is an increasing necessity to deploy autonomous systems in highly heterogeneous, dynamic environments, e.g. service robots in hospitals or autonomous cars on highways. Due to the uncertainty in these environments, the verification results obtained with respect to the system and environment models at design-time might not be transferable to the system behavior at run time. For autonomous systems operating in dynamic environments, safety of motion and collision avoidance are critical requirements. With regard to these requirements, Maček et al. [6] define the passive safety property, which requires that no collision can occur while the autonomous system is moving. To verify this property, we adopt a two phase process which combines static verification methods, used at design time, with dynamic ones, used at run time. In the design phase, we exploit UPPAAL to formalize the autonomous system and its environment as timed automata and the safety property as TCTL formula and to verify the correctness of these models with respect to this property. For the runtime phase, we build a monitor to check whether the assumptions made at design time are also correct at run time. If the current system observations of the environment do not correspond to the initial system assumptions, the monitor sends feedback to the system and the system enters a passive safe state.

Robotics and Integrated Formal Methods: Necessity Meets Opportunity

“…Specifying a monitor to restrict the robotic system to safe behaviours within its environment reduces the verification burden, as only the monitor needs to be verified [21]. For example, a robot's environment can be captured by timed automata and safety properties written in temporal logic [2]. This can be used to build a run-time monitor for the safety properties.…”

Exploiting augmented intelligence in the modeling of safety-critical autonomous systems

“…There is a growing body of research on the verification and testing of AI-based safety-critical autonomous systems. For instance, [AAHR16] presents the formal verification of safety-critical autonomous systems in dynamic environments. The authors adopt a two phase process which combines static verification methods (with UPPAAL), used at design time, with dynamic ones, used at run time (using monitors).…”

Formal Verification for Task Description Languages. A Petri Net Approach

“…Many formalisms have been used to specify or verify robotic systems. Some of the most popular tools specify properties in temporal logic such as UPPAAL [16] and PRISM [17]. Formalisms for discrete-event systems are also among the most widespread and include PNs [18], Time Automata (TA) [19], Finite-State Automata (FSA) [20] and Markov chains [21].…”