Optimization and abstraction: a synergistic approach for analyzing neural network robustness

Abstract

In recent years, the notion of local robustness (or robustness for short) has emerged as a desirable property of deep neural networks. Intuitively, robustness means that small perturbations to an input do not cause the network to perform misclassifications. In this paper, we present a novel algorithm for verifying robustness properties of neural networks. Our method synergistically combines gradient-based optimization methods for counterexample search with abstraction-based proof search to obtain a sound and (δ -)complete decision procedure. Our method also employs a data-driven approach to learn a verification policy that guides abstract interpretation during proof search. We have implemented the proposed approach in a tool called Charon and experimentally evaluated it on hundreds of benchmarks. Our experiments show that the proposed approach significantly outperforms three state-of-the-art tools, namely AI 2 , Reluplex, and Reluval.

NNV: The Neural Network Verification Tool for Deep Neural Networks and Learning-Enabled Cyber-Physical Systems

“…For the "open-loop" verification problem (verification of DNNs), many efficient techniques have been proposed, such as SMT-based methods [22,23,30], mixed-integer linear programming methods [14,24,28], setbased methods [4,17,32,33,48,50,53,57], and optimization methods [51,58]. For the "closed-loop" verification problem (NCCS verification), we note that the Verisig approach [20] is efficient for NNCS with nonlinear plants and with Sigmoid and Tanh activation functions.…”

Abstract Interpretation of Decision Tree Ensemble Classifiers

“…A classifier is stable for some (typically very small) perturbation of its input samples which represents an adversarial attack when it assigns the same class to all the samples within that perturbation, so that impercetible malicious alterations of input objects should not deceive a stable classifier. Formal verification methods for neural networks may rely on a number of different techniques: linear approximation of functions (Weng et al 2018;Zhang et al 2018), semidefinite relaxations (Raghunathan, Steinhardt, and Liang 2018), logical SMT solvers (Huang et al 2017;Katz et al 2017), symbolic interval propagation (Wang et al 2018a), abstract interpretation (Gehr et al 2018;Singh et al 2018; or hybrid synergistic approaches (Anderson et al 2019;Wang et al 2018b). Abstract interpretation (Cousot and Cousot 1977) is a de facto standard technique used since forty years for designing static analysers and verifiers of programming languages.…”

Does fixing bug increase robustness in deep learning?

“…A wide spectrum of studies has been performed, and those include the generation of the adversarial examples [4,8,16,22,23,26,36] and crafting defense [4,10,11,21,28,34,35,42] against such synthetic attacks. Also, studying DNN models as software and utilizing SE techniques have been proven to be useful to validate and test the DNN model [2,15,17,24,27,39,40].…”