Compositional Verification for Autonomous Systems with Deep Learning Components

Păsăreanu, Corina S.; Gopinath, Divya; Yu, Huafeng

doi:10.1007/978-3-319-97301-2_10

Cited by 14 publications

(5 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our work is similar in spirit to the white paper [42] and the work reported in [31] 1 , in that, they propose using abstraction/contracts for perception components. In [31], the authors train generative adversarial networks (GANs) to produce a simpler neural network that abstracts away image sensing and image based perception.…”

Section: Related Workmentioning

confidence: 64%

Verifying Controllers with Convolutional Neural Network-based Perception: A Case for Intelligible, Safe, and Precise Abstractions

Hsieh¹,

Joshi²,

Misailovíc³

et al. 2021

Preprint

View full text Add to dashboard Cite

Convolutional Neural Networks (CNN) for object detection, lane detection, and segmentation now sit at the head of most autonomy pipelines, and yet, their safety analysis remains an important challenge. Formal analysis of perception models is fundamentally difficult because their correctness is hard if not impossible to specify. We present a technique for inferring intelligible and safe abstractions for perception models from system-level safety requirements, data, and program analysis of the modules that are downstream from perception. The technique can help tradeoff safety, size, and precision, in creating abstractions and the subsequent verification. We apply the method to two significant case studies based on highfidelity simulations (a) a vision-based lane keeping controller for an autonomous vehicle and (b) a controller for an agricultural robot. We show how the generated abstractions can be composed with the downstream modules and then the resulting abstract system can be verified using program analysis tools like CBMC. Detailed evaluations of the impacts of size, safety requirements, and the environmental parameters (e.g., lighting, road surface, plant type) on the precision of the generated abstractions suggest that the approach can help guide the search for corner cases and safe operating envelops.

show abstract

Section: Related Workmentioning

confidence: 64%

Verifying Controllers with Convolutional Neural Network-based Perception: A Case for Intelligible, Safe, and Precise Abstractions

Hsieh¹,

Joshi²,

Misailovíc³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Our work is similar in spirit to the idea of using abstraction/contracts for perception components in the white paper by Pȃsȃreanu et al [23]. Following this idea, Katz et al [11] in particular trains generative adversarial networks (GANs) to produce a simpler network.…”

Section: Related Workmentioning

confidence: 90%

Verifying Controllers With Vision-Based Perception Using Safe Approximate Abstractions

Hsieh

Sun

et al. 2022

IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst.

View full text Add to dashboard Cite

Fully formal verification of perception models is likely to remain challenging in the foreseeable future, and yet these models are being integrated into safety-critical control systems. We present a practical method for reasoning about the safety of such systems. Our method is based on systematically constructing approximations of perception models from system-level safety requirements, data, and program analysis of the modules that are downstream from perception. These approximations have some desirable properties like being lowdimensional, intelligible, and tractable. The closed-loop system, with the approximation substituting the actual perception model, is verified to be safe. Establishing formal relationship between the actual and the approximate perception models remains well beyond available verification techniques. However, we do provide a useful empirical measure of their closeness called precision. Overall, our method can trade off the size of the approximation against precision. We apply the method to two significant case studies (a) a vision-based lane tracking controller for an autonomous vehicle and (b) a controller for an agricultural robot. We show how the generated approximations for each system can be composed with the downstream modules and be verified using program analysis tools like CBMC. Detailed evaluations of the impacts of size, and the environmental parameters (e.g., lighting, road surface, plant type) on the precision of the generated approximations suggest that the approach can be useful for realistic control systems.

show abstract

“…The clusters helps to better model errors for regions in which perception may not be accurate. In fact, clustering methods has been used for analysis of perception components in [15], but not in the context of control design.…”

Section: D K P C T 8 O I R 5 K H U Y 2 6 4 = " >mentioning

confidence: 99%

Counterexample-Guided Synthesis of Perception Models and Control

Ghosh¹,

Pant²,

Ravanbakhsh³

et al. 2019

Preprint

View full text Add to dashboard Cite

We consider the problem of synthesizing safe and robust controllers for real world robotic systems like autonomous vehicles, which rely on complex perception modules. We propose a counterexample-guided synthesis framework which iteratively learns perception models that enable finding safe control policies. We use counterexamples to extract information relevant for modeling the errors in perception modules. Such models then can be used to synthesize controllers robust to errors in perception. If the resulting policy is not safe, we gather new counterexamples. By repeating the process, we eventually find a controller which can keep the system safe even when there is perception failure. Finally, we show that our framework computes robust controllers for autonomous vehicles in two different simulated scenarios: (i) lane keeping, and (ii) automatic braking.

show abstract

Compositional Verification for Autonomous Systems with Deep Learning Components

Cited by 14 publications

References 16 publications

Verifying Controllers with Convolutional Neural Network-based Perception: A Case for Intelligible, Safe, and Precise Abstractions

Verifying Controllers with Convolutional Neural Network-based Perception: A Case for Intelligible, Safe, and Precise Abstractions

Verifying Controllers With Vision-Based Perception Using Safe Approximate Abstractions

Counterexample-Guided Synthesis of Perception Models and Control

Contact Info

Product

Resources

About