Symbolic Execution for Importance Analysis and Adversarial Generation in Neural Networks

Gopinath, Divya; Zhang, Mengshi; Wang, Kaiyuan; Kadron, Ismet Burak; Păsăreanu, Corina S.; Khurshid, Sarfraz

doi:10.1109/issre.2019.00039

Cited by 15 publications

(11 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We propose two mechanism for this. Our first approach translates the deep learning program into an imperative program [20], [24]. Probes are inserted in this imperative program to capture and save model variables such as weights, learning rate gra dients during training.…”

Section: A P P R O a C Hmentioning

confidence: 99%

“…The closest related work in terms of technical ideas is by Gopinath et al [20], [24]. Gopinath et al proposed a new approach (DeepCheck) inspired from program analysis to test a Deep Neural Network (DNN) using symbolic execution.…”

Section: I Re L a T E D W O R Kmentioning

confidence: 99%

“…We propose two techniques. The first technique, inspired by [20], translates the code into an intermediate form which we call imperative representation o f the DNN. The purpose of the imperative representation is to make certain (ensure) that internal states of the DNN are observable, thus our method uses a white box approach.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

DeepLocalize: Fault Localization for Deep Neural Networks

Wardat

Rajan

2021

2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)

View full text Add to dashboard Cite

Deep neural networks (DNNs) are becoming an integral part of most software systems. Previous work has shown that DNNs have bugs. Unfortunately, existing debugging techniques don t support localizing DNN bugs because of the lack of understanding of model behaviors. The entire DNN model appears as a black box.To address these problems, we propose an approach and a tool that automatically determines whether the model is buggy or not, and identifies the root causes for DNN errors. Our key insight is that historic trends in values propagated between layers can be analyzed to identify faults, and also localize faults. To that end, we first enable dynamic analysis of deep learning applications: by converting it into an imperative representation and alternatively using a callback mechanism. Both mechanisms allows us to insert probes that enable dynamic analysis over the traces produced by the DNN while it is being trained on the training data. We then conduct dynamic analysis over the traces to identify the faulty layer or hyperparameter that causes the error. We propose an algorithm for identifying root causes by capturing any numerical error and monitoring the model during training and finding the relevance of every layer/parameter on the DNN outcome. We have collected a benchmark containing 40 buggy models and patches that contain real errors in deep learning applications from Stack Overflow and GitHub. Our benchmark can be used to evaluate automated debugging tools and repair techniques. We have evaluated our approach using this DNN bug-and-patch benchmark, and the results showed that our approach is much more effective than the existing debugging approach used in the state-of-the-practice Kerns library. For 34/40 cases, our approach was able to detect faults whereas the best debugging approach provided by Kerns detected 32/40 faults. Our approach was able to localize 21/40 bugs whereas Keras did not localize any faults.

show abstract

Section: A P P R O a C Hmentioning

confidence: 99%

Section: I Re L a T E D W O R Kmentioning

confidence: 99%

See 1 more Smart Citation

DeepLocalize: Fault Localization for Deep Neural Networks

Wardat

Rajan

2021

2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)

View full text Add to dashboard Cite

show abstract

“…Prior work on DNN verification focused on safety and robustness [4,6,11,15,18,31,34,35,52]. More recent research tackles testing of DNNs [23,44,45,47,50,56,58,60]. Our algorithms can fix errors found by such tools.…”

Section: Related Workmentioning

confidence: 99%

Provable Repair of Deep Neural Networks

Sotoudeh¹,

Thakur²

2021

Preprint

View full text Add to dashboard Cite

Deep Neural Networks (DNNs) have grown in popularity over the past decade and are now being used in safety-critical domains such as aircraft collision avoidance. This has motivated a large number of techniques for finding unsafe behavior in DNNs. In contrast, this paper tackles the problem of correcting a DNN once unsafe behavior is found. We introduce the provable repair problem, which is the problem of repairing a network 𝑁 to construct a new network 𝑁 ′ that satisfies a given specification. If the safety specification is over a finite set of points, our Provable Point Repair algorithm can find a provably minimal repair satisfying the specification, regardless of the activation functions used. For safety specifications addressing convex polytopes containing infinitely many points, our Provable Polytope Repair algorithm can find a provably minimal repair satisfying the specification for DNNs using piecewise-linear activation functions. The key insight behind both of these algorithms is the introduction of a Decoupled DNN architecture, which allows us to reduce provable repair to a linear programming problem. Our experimental results demonstrate the efficiency and effectiveness of our Provable Repair algorithms on a variety of challenging tasks. CCS Concepts: • Computing methodologies → Neural networks; • Theory of computation → Linear programming; • Software and its engineering → Software postdevelopment issues.

show abstract

“…We propose two techniques. The first technique, inspired by [20], translates the code into an intermediate form which we call imperative representation of the DNN. The purpose of the imperative representation is to make certain (ensure) that internal states of the DNN are observable, thus our method uses a white box approach.…”

Section: Introductionmentioning

confidence: 99%

DeepLocalize: Fault Localization for Deep Neural Networks

Wardat¹,

Le²,

Rajan³

2021

Preprint

View full text Add to dashboard Cite

Deep neural networks (DNNs) are becoming an integral part of most software systems. Previous work has shown that DNNs have bugs. Unfortunately, existing debugging techniques don't support localizing DNN bugs because of the lack of understanding of model behaviors. The entire DNN model appears as a black box. To address these problems, we propose an approach and a tool that automatically determines whether the model is buggy or not, and identifies the root causes for DNN errors. Our key insight is that historic trends in values propagated between layers can be analyzed to identify faults, and also localize faults. To that end, we first enable dynamic analysis of deep learning applications: by converting it into an imperative representation and alternatively using a callback mechanism. Both mechanisms allows us to insert probes that enable dynamic analysis over the traces produced by the DNN while it is being trained on the training data. We then conduct dynamic analysis over the traces to identify the faulty layer or hyperparameter that causes the error. We propose an algorithm for identifying root causes by capturing any numerical error and monitoring the model during training and finding the relevance of every layer/parameter on the DNN outcome. We have collected a benchmark containing 40 buggy models and patches that contain real errors in deep learning applications from Stack Overflow and GitHub. Our benchmark can be used to evaluate automated debugging tools and repair techniques. We have evaluated our approach using this DNN bug-and-patch benchmark, and the results showed that our approach is much more effective than the existing debugging approach used in the state-of-the-practice Keras library. For 34/40 cases, our approach was able to detect faults whereas the best debugging approach provided by Keras detected 32/40 faults. Our approach was able to localize 21/40 bugs whereas Keras did not localize any faults.

show abstract

Symbolic Execution for Importance Analysis and Adversarial Generation in Neural Networks

Cited by 15 publications

References 23 publications

DeepLocalize: Fault Localization for Deep Neural Networks

DeepLocalize: Fault Localization for Deep Neural Networks

Provable Repair of Deep Neural Networks

DeepLocalize: Fault Localization for Deep Neural Networks

Contact Info

Product

Resources

About