Verification of Image-based Neural Network Controllers Using Generative Models

Abstract

Neural networks are often used to process information from image-based sensors to produce control actions. While they are effective for this task, the complex nature of neural networks makes their output difficult to verify and predict, limiting their use in safety-critical systems. For this reason, recent work has focused on combining techniques in formal methods and reachability analysis to obtain guarantees on the closedloop performance of neural network controllers. However, these techniques do not scale to the high-dimensional and complicated input space of image-based neural network controllers. In this work, we propose a method to address these challenges by training a generative adversarial network (GAN) to map states to plausible input images. By concatenating the generator network with the control network, we obtain a network with a lowdimensional input space. This insight allows us to use existing closed-loop verification tools to obtain formal guarantees on the performance of image-based controllers. We apply our approach to provide safety guarantees for an image-based neural network controller for an autonomous aircraft taxi problem. We guarantee that the controller will keep the aircraft on the runway and guide the aircraft towards the center of the runway. The guarantees we provide are with respect to the set of input images modeled by our generator network, so we provide a recall metric to evaluate how well the generator captures the space of plausible images.

Verifying Controllers with Convolutional Neural Network-based Perception: A Case for Intelligible, Safe, and Precise Abstractions

“…Our work is similar in spirit to the white paper [42] and the work reported in [31] 1 , in that, they propose using abstraction/contracts for perception components. In [31], the authors train generative adversarial networks (GANs) to produce a simpler neural network that abstracts away image sensing and image based perception. The simpler network directly transforms states and environment parameters to estimates similar to our abstraction 𝑀.…”

“…Analysis of closed loop systems with NNs. The closest related works are VerifAI [13] and a recent work [31]. VerifAI [13] and related publications provide a comprehensive framework to analyze a closed loop system with ML-based perception components.…”

“…Not coincidentally, research on formally verifying isolated neural networks (NN), has received a degree of attention [6,15,26,30,46,51]. System-level safety assurance brings a somewhat different set of opportunities and challenges and has been addressed in a small number of recent efforts [12,13,31].…”

Synthesizing Adversarial Visual Scenarios for Model-Based Robotic Control

“…We use a public dataset from the photo-realistic X-Plane simulator [1] consisting of images from the plane's wingmounted camera in diverse weather conditions and at various poses on the runway. This standard benchmark dataset has been used in recent works on robust and verified perception [19], [29], [20]. Our prime goal is to test that our adversarial training method yields models that outperform benchmarks at generalizing to challenging OoD weather conditions.…”