Symbolic Execution for Attribution and Attack Synthesis in Neural Networks

Abstract

Deep Neural Networks (DNN) are increasingly used in a variety of applications, many of them with substantial safety and security concerns. This paper introduces DeepCheck, a new approach for validating DNNs based on core ideas from program analysis, specifically from symbolic execution. The idea is to translate a DNN into an imperative program, thereby enabling program analysis to assist with DNN validation. A basic translation however creates programs that are very complex to analyze. DeepCheck introduces novel techniques for lightweight symbolic analysis of DNNs and applies them in the context of image classification to address two challenging problems in DNN analysis: 1) identification of important pixels (for attribution and adversarial generation); and 2) creation of 1-pixel and 2-pixel attacks. Experimental results using the MNIST data-set show that DeepCheck's lightweight symbolic analysis provides a valuable tool for DNN validation.

Machine Learning Testing: Survey, Landscapes and Horizons

“…When applying symbolic execution on the machine learning code, there comes many challenges. Gopinath [101] listed three such challenges for neural networks in their paper, which work for other ML modes as well: (1) the networks have no explicit branching; (2) the networks may be highly non-linear, with no well-developed solvers for constraints; and (3) there are scalability issues because the structure of the ML models are usually very complex and are beyond the capabilities of current symbolic reasoning tools.…”

“…Considering these challenges, Gopinath [101] introduced DeepCheck. It transforms a Deep Neural Network (DNN) into a program to enable symbolic execution to find pixel attacks that have the same activation pattern as the original image.…”

“…Other techniques to generate test data to check the robustness of neural networks include symbolic execution [101], [104], fuzz testing [83], combinatorial Testing [141], and abstract interpretation [182]. In Section 5.1, we introduce more contents about test generation techniques.…”

Software Assurance in an Uncertain World

“…For their successful adoption in society, we need to ensure that they are trustworthy, including obtaining confidence in their behavior and robustness. Significant strides have already been made in this space, from extending mature testing and verification techniques to reasoning about neural networks [24,37,47,54] for properties such as safety, robustness and adequate handling of adversarial examples [26,34]. There is active work in designing systems that balance learning under uncertainty and acting safely, e.g., [52] as well as the broad notion of fairness and explainability in AI, e.g., [49].…”

DeepFault: Fault Localization for Deep Neural Networks

“…Recent research in the DNN testing area introduces novel white-box and black-box techniques for testing DNNs [20,28,36,37,48,54,55]. Some techniques transform valid training data into adversarial through mutation-based heuristics [65], apply symbolic execution [15], combinatorial [37] or concolic testing [55], while others propose new DNN-specific coverage criteria, e.g., neuron coverage [48] and its variants [35] or MC/DC-inspired criteria [52]. We review related work in Section 6.…”