Verifying Quantized Neural Networks using SMT-Based Model Checking

Sena, Luiz; Song, Xidan; Alves, Erickson; Bessa, Iury; Manino, Edoardo; Cordeiro, Lucas C.

doi:10.48550/arxiv.2106.05997

Cited by 2 publications

(2 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(Baranowski et al 2020) introduced fixed-point arithmetic SMT to verify QNNs. The works of (Sena et al 2021(Sena et al , 2022 have studied SMT-based verification for QNNs as well. Recently, (Mistry, Saha, and Biswas 2022) proposed encoding of the QNN verification problem into a mixed-integer linear programming (MILP) instance.…”

Section: Related Workmentioning

confidence: 99%

Quantization-Aware Interval Bound Propagation for Training Certifiably Robust Quantized Neural Networks

Lechner

Žikelić

Chatterjee

et al. 2023

AAAI

View full text Add to dashboard Cite

We study the problem of training and certifying adversarially robust quantized neural networks (QNNs). Quantization is a technique for making neural networks more efficient by running them using low-bit integer arithmetic and is therefore commonly adopted in industry. Recent work has shown that floating-point neural networks that have been verified to be robust can become vulnerable to adversarial attacks after quantization, and certification of the quantized representation is necessary to guarantee robustness. In this work, we present quantization-aware interval bound propagation (QA-IBP), a novel method for training robust QNNs. Inspired by advances in robust learning of non-quantized networks, our training algorithm computes the gradient of an abstract representation of the actual network. Unlike existing approaches, our method can handle the discrete semantics of QNNs. Based on QA-IBP, we also develop a complete verification procedure for verifying the adversarial robustness of QNNs, which is guaranteed to terminate and produce a correct answer. Compared to existing approaches, the key advantage of our verification procedure is that it runs entirely on GPU or other accelerator devices. We demonstrate experimentally that our approach significantly outperforms existing methods and establish the new state-of-the-art for training and certifying the robustness of QNNs.

show abstract

Section: Related Workmentioning

confidence: 99%

Quantization-Aware Interval Bound Propagation for Training Certifiably Robust Quantized Neural Networks

Lechner

Žikelić

Chatterjee

et al. 2023

AAAI

View full text Add to dashboard Cite

show abstract

“…While Reluplex only applies to feedforward neural networks with ReLU activation functions, the technique was extended to fully connected and convolutional neural networks with arbitrary piecewise-linear activation functions with Marabou [14]. Other works in SMT-based verification include [15], [16]. Related to SMT-solving has been the use of mixed-integer programming such as in [17], [18], [19].…”

Section: Introductionmentioning

confidence: 99%

Parametric Chordal Sparsity for SDP-based Neural Network Verification

Xue¹,

Lindemann²,

Alur³

2022

Preprint

View full text Add to dashboard Cite

Many future technologies rely on neural networks, but verifying the correctness of their behavior remains a major challenge. It is known that neural networks can be fragile in the presence of even small input perturbations, yielding unpredictable outputs. The verification of neural networks is therefore vital to their adoption, and a number of approaches have been proposed in recent years. In this paper we focus on semidefinite programming (SDP) based techniques for neural network verification, which are particularly attractive because they can encode expressive behaviors while ensuring a polynomialtime decision. Our starting point is the DeepSDP framework proposed by Fazlyab et al, which uses quadratic constraints to abstract the verification problem into a large-scale SDP. When the size of the neural network grows, however, solving this SDP quickly becomes intractable. Our key observation is that by leveraging chordal sparsity and specific parametrizations of DeepSDP, we can decompose the primary computational bottleneck of DeepSDP -a large linear matrix inequality (LMI) -into an equivalent collection of smaller LMIs. Our parametrization admits a tunable parameter, allowing us to trade-off efficiency and accuracy in the verification procedure. We call our formulation Chordal-DeepSDP, and provide experimental evaluation to show that it can: (1) effectively increase accuracy with the tunable parameter and (2) outperform DeepSDP on deeper networks.

show abstract

Verifying Quantized Neural Networks using SMT-Based Model Checking

Cited by 2 publications

References 47 publications

Quantization-Aware Interval Bound Propagation for Training Certifiably Robust Quantized Neural Networks

Quantization-Aware Interval Bound Propagation for Training Certifiably Robust Quantized Neural Networks

Parametric Chordal Sparsity for SDP-based Neural Network Verification

Contact Info

Product

Resources

About