NNV: The Neural Network Verification Tool for Deep Neural Networks and Learning-Enabled Cyber-Physical Systems

Abstract

This paper presents the Neural Network Verification (NNV) software tool, a set-based verification framework for deep neural networks (DNNs) and learning-enabled cyber-physical systems (CPS). The crux of NNV is a collection of reachability algorithms that make use of a variety of set representations, such as polyhedra, star sets, zonotopes, and abstract-domain representations. NNV supports both exact (sound and complete) and over-approximate (sound) reachability algorithms for verifying safety and robustness properties of feed-forward neural networks (FFNNs) with various activation functions. For learning-enabled CPS, such as closed-loop control systems incorporating neural networks, NNV provides exact and over-approximate reachability analysis schemes for linear plant models and FFNN controllers with piecewise-linear activation functions, such as ReLUs. For similar neural network control systems (NNCS) that instead have nonlinear plant models, NNV supports overapproximate analysis by combining the star set analysis used for FFNN controllers with zonotope-based analysis for nonlinear plant dynamics building on CORA. We evaluate NNV using two real-world case studies: the first is safety verification of ACAS Xu networks, and the second deals with the safety verification of a deep learning-based adaptive cruise control system.

ARCH-COMP20 Category Report: Artificial Intelligence and Neural Network Control Systems (AINNCS) for Continuous and Hybrid Systems Plants

“…NNV NNV (Neural Network Verification Tool) [30,28,29,35,37,34,33,31,36,1] is a Matlab toolbox that implements reachability analysis methods for neural network verification, with a particular focus on applications of closed-loop neural network control systems in autonomous cyber-physical systems. NNV uses a star-set state-space representation and reachability algorithm that allows for a layer-by-layer computation of exact or overapproximate reachable sets for feed-forward and convolutional neural networks.…”

Verification of Closed-loop Systems with Neural Network Controllers

“…However, the vast majority of these techniques have only been able to deal with feed-forward neural networks with piecewise-linear activation functions [4]. Additionally, the bulk of these methods have primarily considered the verification of input-output properties of neural networks in isolation [22], and there are only a handful of works that have explicitly addressed the verification of closed-loop control systems with neural network controllers [5,8,[19][20][21]. One of the central challenges in verifying neural network control systems is that applying existing methodology to these systems is not straightforward [9], and a simple combination of verification tools for non-linear ordinary differential equations along with a neural network reachability tool suffers from severe overestimation errors [5].…”

“…The benchmarks elucidated in this paper are modeled using Simulink/Stateflow and we have also used the Hybrid Source Transformer tool (HYST) 2 [1] to transform the models of the plants into the SpaceEx format [6]. Additionally, we have also provided the neural network controllers, in a variety of formats including a matlab format used in the (Neural Network Verication) NNV 3 framework proposed by Tran et al [19], Simulink models, and the Open Neural Network Exchange 4 (ONNX) format. The following section presents a brief description of the benchmarks presented for verification.…”

“…In this section we present a brief investigation of the verification of the Adaptive Cruise Controller (ACC) Benchmark in order to demonstrate the verification of a closed loop neural network control system. This experiment was originally proposed in [19] and we present the results here as an example of the methods currently available within the research literature. As previously mentioned, the safety verification problem of interest is that when the ego car makes use of the speed control mode and both the lead car and the ego car are driving with a safe distance between them, if the lead car suddenly brakes, we expect that the neural network controllers will ensure that the ego car brakes correspondingly so as to maintain a safe distance between the two cars [19].…”

“…This experiment was originally proposed in [19] and we present the results here as an example of the methods currently available within the research literature. As previously mentioned, the safety verification problem of interest is that when the ego car makes use of the speed control mode and both the lead car and the ego car are driving with a safe distance between them, if the lead car suddenly brakes, we expect that the neural network controllers will ensure that the ego car brakes correspondingly so as to maintain a safe distance between the two cars [19]. In this particular case we consider a neural network controller with 5 hidden layers each consisting of 20 neurons and make use of the Neural Network Verification Toolbox (NNV) proposed by Tran et al [19] to perform the verification.…”