Robustness of Neural Networks: A Probabilistic and Practical Approach

Abstract

Neural networks are becoming increasingly prevalent in software, and it is therefore important to be able to verify their behavior. Because verifying the correctness of neural networks is extremely challenging, it is common to focus on the verification of other properties of these systems. One important property, in particular, is robustness. Most existing definitions of robustness, however, focus on the worst-case scenario where the inputs are adversarial. Such notions of robustness are too strong, and unlikely to be satisfied by-and verifiable forpractical neural networks. Observing that real-world inputs to neural networks are drawn from non-adversarial probability distributions, we propose a novel notion of robustness: probabilistic robustness, which requires the neural network to be robust with at least (1 − ) probability with respect to the input distribution. This probabilistic approach is practical and provides a principled way of estimating the robustness of a neural network. We also present an algorithm, based on abstract interpretation and importance sampling, for checking whether a neural network is probabilistically robust. Our algorithm uses abstract interpretation to approximate the behavior of a neural network and compute an overapproximation of the input regions that violate robustness. It then uses importance sampling to counter the effect of such overapproximation and compute an accurate estimate of the probability that the neural network violates the robustness property.

Machine Learning Testing: Survey, Landscapes and Horizons

“…Other techniques to generate test data to check the robustness of neural networks include symbolic execution [101], [104], fuzz testing [83], combinatorial Testing [141], and abstract interpretation [182]. In Section 5.1, we introduce more contents about test generation techniques.…”

Improving Diversity and Quality of Adversarial Examples in Adversarial Transformation Network

“…These state-based metrics could be combined with L p -norm. Several state-based metrics could be referred to adversarial distance (Papernot et al, 2015), average robustness (Moosavi-Dezfooli et al, 2015), adversarial severity (Bastani et al, 2016), adversarial frequency (Bastani et al, 2016), point-wise robustness (Bastani et al, 2016), local/global adversarial robustness (Katz et al, 2017), neuron coverage (Pei et al, 2017), a set of multigranularity testing criteria (Ma et al, 2018), Lipschitz continuity (Sun et al, 2018b), sign-sign coverage (Sun et al, 2018a), and probabilistic robustness (Mangal et al, 2019 Alcorn et al (2018) apply natural transformation such as changing viewpoint, lighting, coloring. Naderi et al (2021) propose a method based on geometric transformations to generate natural perturbations.…”

Improving Neural Network Verification through Spurious Region Guided Refinement

“…They follow a statistical way to estimate the label change rate of an input, which motivates us to give a formal definition of the property showing a low label change rate, and to consider the verification problem for such a property. Below we recall the definition of quantitative robustness [27], where we have a parameter 0 < η ≤ 1 representing the confidence of robustness.…”