Decentralized Real-Time Safety Verification for Distributed Cyber-Physical Systems

Tran, Hoang-Dung; Nguyen, Luan Viet; Musau, Patrick; Xiang, Weiming; Johnson, Taylor T.

doi:10.1007/978-3-030-21759-4_15

Cited by 15 publications

(8 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our future work is extending the proposed methods for nonlinear NNCS with other types of nonlinear activation functions such as Tanh or Sigmoid. We are also investigating the runtime verification for CPS with learning-enabled components leveraging the recent promising approach proposed in [32] in combination with the neural network reachability analysis methods.…”

Section: Discussionmentioning

confidence: 99%

Safety Verification of Cyber-Physical Systems with Reinforcement Learning Control

Tran

Cai

Diego

et al. 2019

ACM Trans. Embed. Comput. Syst.

Self Cite

View full text Add to dashboard Cite

This paper proposes a new forward reachability analysis approach to verify safety of cyber-physical systems (CPS) with reinforcement learning controllers. The foundation of our approach lies on two efficient, exact and over-approximate reachability algorithms for neural network control systems using star sets, which is an efficient representation of polyhedra. Using these algorithms, we determine the initial conditions for which a safety-critical system with a neural network controller is safe by incrementally searching a critical initial condition where the safety of the system cannot be established. Our approach produces tight over-approximation error and it is computationally efficient, which allows the application to practical CPS with learning enable components (LECs). We implement our approach in NNV, a recent verification tool for neural networks and neural network control systems, and evaluate its advantages and applicability by verifying safety of a practical Advanced Emergency Braking System (AEBS) with a reinforcement learning (RL) controller trained using the deep deterministic policy gradient (DDPG) method. The experimental results show that our new reachability algorithms are much less conservative than existing polyhedra-based approaches. We successfully determine the entire region of the initial conditions of the AEBS with the RL controller such that the safety of the system is guaranteed, while a polyhedra-based approach cannot prove the safety properties of the system.

show abstract

Section: Discussionmentioning

confidence: 99%

Safety Verification of Cyber-Physical Systems with Reinforcement Learning Control

Tran

Cai

Diego

et al. 2019

ACM Trans. Embed. Comput. Syst.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Form of Reachability Analysis Type of Guarantee [87], [88] maxFRS + minBRS guarantees collision if no solution for "anticipated reachable set" is found [89] FRS of controlled dynamics predicts unavoidable collision within control policy [90] overapproximated maxFRS, maxFRT all possible forward motions are included [81] inaccuracy around trajectory-planned vehicle checks probability of collision within the inaccuracy model bounds [91] minBRT guarantees collision free in horizon [72] maxBRS for partially-controlled & uncontrolled vehicle conservative collision-free guarantee for non-extreme driving [92] overapproximated maxFRI guarantee "not-at-fault" safety by checking potential collision in time intervals leading to end of time horizon [93] maxBRT guarantees safe interoperability: possibly safe human takeover from autonomous driving system [94] maxFRS of uncontrolled dynamics guarantees collision free in horizon [95] funnel (a variant of FRT for controlled vehicle) guarantees collision-free if a funnel can be found to stay clear of obstacles at all times [96], [97], [98] maxFRS implemented by Flow* [99] guarantees safety of deep neural network (DNN) controlled close-loop system [100] maxFRS of (possibly) occluded vehicle guarantees collision-free with possibly occluded vehicle(s) [101] maxFRS of each agent communicated through a decentralized network real-time collision-free guarantee for a group of autonomous agents [102] maxFRS + collision state pruning + maxBRS guarantees collision-free and successful reach of target state if "controller contract" (state constraint tubes) are honored [86] maxFRT + pedestrian intent prediction guarantees safety within a high probability bound of pedestrian motion collision frequency [53], a vital quantity in determining automotive safety integritity level (ASIL) in ISO26262 [48]. If the vehicle is indeed in such a state, an impact preparation should be executed (usually with extensive use of braking to slow down [104], or by following not-at-fault trajectories [92].)…”

Section: Referencementioning

confidence: 99%

Formal Certification Methods for Automated Vehicle Safety Assessment

Zhao,

Yurtsever,

Paulson

et al. 2022

Preprint

View full text Add to dashboard Cite

Challenges related to automated driving are no longer focused on just the construction of such automated vehicles (AVs), but in assuring the safety of their operation. Recent advances in Level 3 and Level 4 autonomous driving have motivated more extensive study in safety guarantees of complicated AV maneuvers, which aligns with the goal of ISO 21448 (Safety of the Intended Functions, or SOTIF), i.e. minimizing unsafe scenarios both known and unknown, as well as Vision Zero -eliminating highway fatalities by 2050. A majority of approaches used in providing safety guarantees for AV motion control originate from formal methods, especially reachability analysis (RA), which relies on mathematical models for the dynamic evolution of the system to provide guarantees. However, to the best of the authors' knowledge, there have been no review papers dedicated to describing and interpreting state-of-the-art of formal methods in the context of AVs. In this work, we provide both an overview of the safety verification, validation and certification process, as well as review formal safety techniques that are best suited to AV applications. We also propose a unified scenario coverage framework that can provide either a formal or sample-based estimate of safety verification for full AVs. Finally, remaining challenges and future opportunities beyond the scope of current published research for assured AV safety are presented.

show abstract

“…Existing techniques on real-time assurance [44][45][46][47][48][49] are largely limited to a system where the plant dynamics are known and simple, and environment assumptions are static. However, we consider a system that includes legacy components that have complex and unknown dynamics, so existing real-time verification approaches for CPSs are computationally expensive and cannot be applied to those systems in an online manner.…”

Section: Real-time Assurance Of Cpsmentioning

confidence: 99%

Towards a framework of enforcing resilient operation of cyber‐physical systems with unknown dynamics

Nguyen

Gupta

2021

IET Cyber-Phy Sys Theory & Ap

View full text Add to dashboard Cite

Ensuring that safety-critical cyber-physical systems (CPSs) continue to satisfy correctness and safety specifications even under faults or adversarial attacks is very challenging, especially in the presence of legacy components for which accurate models are unknown to the designer. Current techniques for secure-by-design systems engineering do not provide an end-to-end methodology for a designer to provide real-time assurance for safety-critical CPSs by identifying system dynamics and updating control strategies in response to newly discovered faults, attacks or other changes such as system upgrades. We propose a new methodology, along with an integrated framework implemented in MATLAB to guarantee the resilient operation of safety-critical CPSs with unknown dynamics. The proposed framework consists of three main components. The runtime monitor evaluates the system behaviour on-the-fly against its correctness specifications expressed as signal temporal logic formulas. The model synthesiser incorporates a sparse identification approach that is used to continually update the plant model and control policies to adapt to any changes in the system or the environment. The decision and control module designs a controller to ensure that the correctness specifications are satisfied at runtime. For evaluation, we apply our proposed framework to ensure the resilient operations of two CPS case studies.This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

show abstract

Decentralized Real-Time Safety Verification for Distributed Cyber-Physical Systems

Cited by 15 publications

References 27 publications

Safety Verification of Cyber-Physical Systems with Reinforcement Learning Control

Safety Verification of Cyber-Physical Systems with Reinforcement Learning Control

Formal Certification Methods for Automated Vehicle Safety Assessment

Towards a framework of enforcing resilient operation of cyber‐physical systems with unknown dynamics

Contact Info

Product

Resources

About