Tightening Robustness Verification of Convolutional Neural Networks with Fine-Grained Linear Approximation

Abstract

The robustness of neural networks can be quantitatively indicated by a lower bound within which any perturbation does not alter the original input’s classification result. A certified lower bound is also a criterion to evaluate the performance of robustness verification approaches. In this paper, we present a tighter linear approximation approach for the robustness verification of Convolutional Neural Networks (CNNs). By the tighter approximation, we can tighten the robustness verification of CNNs, i.e., proving they are robust within a larger 10 perturbation distance. Furthermore, our approach is applicable to general sigmoid-like activation functions. We implement DeepCert, the resulting verification toolkit. We evaluate it with open-source benchmarks, including LeNet and the models trained on MNIST and CIFAR. Experimental results show that DeepCert outperforms other state-of-the-art robustness verification tools with at most 286.28% improvement to the certified lower bound and 1566.76 times speedup for the same neural networks.

Provably Tightest Linear Approximation for Robustness Verification of Sigmoid-like Neural Networks

“…As directly computing 𝜖 is difficult due to the non-linearity of the constraint (1), most of the state-of-the-art approaches [5,51,55] adopt the efficient binary search algorithm to first determine a candidate 𝜖 and then check whether (1) is true or false on 𝜖.…”

“…Apparently, 𝑓 𝐿,𝑠 0 (𝑥) − 𝑓 𝑈 ,𝑠 (𝑥) > 0 is a sufficient condition of Formula (1), and it is significantly more efficient to prove or disprove. Therefore, it is widely adopted in neural network verification [5,18,26,51], although it may produce false positives when it is disproved. Definition 2 (Upper/Lower linear bounds).…”

“…For example, a larger certified lower robust bound [5,28] (the perturbation distance under which a neural network is proved robust against any allowable perturbation) is preferable in approximation. Several characterizations of tightness and approximation approaches have been proposed for Sigmoid-like activation functions [5,18,26,28,51,55]. However, they are all heuristic and lack theoretical foundations for the individual outperformance.…”

“…We have implemented a prototype of our approach called NeWise 1 and extensively compared it to three state-of-the-art tools, namely DeepCert [51], VeriNet [18], and RobustVerifier [26]. Our experimental results show that NeWise (i) achieves up to 251.28% improvement to certified lower robustness bounds in the provably tightest cases and (ii) exhibits up to 122.22% improvement to certified lower robustness bounds on convolutional networks.…”

Fine-Grained Neural Network Abstraction for Efficient Formal Verification

“…Another class of abstraction-based neural network verification approaches rely on the abstraction of the constraints transformed from original neural networks, but not the abstraction of neural networks. Representative approaches include abstraction interpretation [9,14] and linear relaxation and overapproximation [4,19]. After abstraction, they resort to efficient linear programming solvers such as SMT and MILP solvers to check the satisfiability of abstracted constraints.…”