Verifying Equivalence Properties of Neural Networks with ReLU Activation Functions

Büning, Marko Kleine; Kern, Philipp; Sinz, Carsten

doi:10.1007/978-3-030-58475-7_50

Cited by 7 publications

(4 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unfortunately, quantization-aware training may break the assumptions of the existing NNE verification tools [11][12][13]39]. Thus, in this paper, we focus on post-training quantization.…”

Section: B Quantization-aware Trainingmentioning

confidence: 99%

“…In addition to those, we mention in Section II, all the following are viable alternatives. First, Büning et al [11] defined the notion of relaxed equivalence, because exact equivalence (see Section II-D) is hard to solve. They choose to encode equivalence properties into MILP.…”

Section: Neural Network Equivalencementioning

confidence: 99%

“…Indeed, specific inputs may exist for which a network's performance degrades significantly [5,10]. Consequently, the only way to guarantee accuracy for a QNN is reformulating the problem under the notion of equivalence checking (EC) [11][12][13]. This property states that two NN are equivalent when they produce similar outputs for inputs in a given domain [12,13].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Counterexample Guided Neural Network Quantization Refinement

Matos,

de Lima Filho,

Bessa

et al. 2024

IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst.

View full text Add to dashboard Cite

Deploying Neural networks (NNs) in low-resource domains is challenging because of their high computing, memory, and power requirements. For this reason, NNs are often quantized before deployment, but such an approach degrades their accuracy. Thus, we propose the counterexample guided neural network quantization refinement (CEG4N) framework, which combines search-based quantization and equivalence checking. The former minimizes computational requirements, while the latter guarantees that the behavior of an NN does not change after quantization. We evaluate CEG4N on a diverse set of benchmarks, including large and small NNs. Our technique successfully quantizes the networks in the chosen evaluation set, while producing models with up to 163% better accuracy than state-of-the-art techniques.

show abstract

“…Unfortunately, quantization-aware training may break the assumptions of the existing NNE verification tools [11][12][13]39]. Thus, in this paper, we focus on post-training quantization.…”

Section: B Quantization-aware Trainingmentioning

confidence: 99%

Section: Neural Network Equivalencementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Counterexample Guided Neural Network Quantization Refinement

Matos,

de Lima Filho,

Bessa

et al. 2024

IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst.

View full text Add to dashboard Cite

show abstract

“…To perform calculations within each neuron, we employed the rectified linear unit (ReLU) function as the activation function, which is the most widely used activation function [31]. The ReLU function produces an output of zero if the input value is less than zero; otherwise, it is equal to the provided input value.…”

Section: Ann Modelmentioning

confidence: 99%

Blood Oxygen Saturation Estimation with Laser-Induced Graphene Respiration Sensor

Madevska Bogdanova,

Koteska,

Vićentić

et al. 2024

Journal of Sensors

View full text Add to dashboard Cite

Measuring blood oxygen saturation (SpO2) is crucial in a triage process for identifying patients with respiratory distress or shock, since low SpO2 levels indicate compromised hemostability and the need for priority treatment. This paper explores the use of wearable mechanical deflection sensors based on laser-induced graphene (LIG) for SpO2 estimation. The LIG sensors are attached to a subject’s chest for real-time monitoring of respiratory signals. We have developed a novel database of the respiratory signals, with corresponding SpO2 values ranging from 86% to 100%. The database is used to develop an artificial neural network model for SpO2 estimation. The neural network performance is promising, with regression metrics mean squared error = 0.184, mean absolute error = 0.301, root mean squared error = 0.429, and R-squared = 0.804. The use of mechanical respiration sensors in combination with neural networks in biosensing opens new possibilities for noninvasive SpO2 monitoring and other innovative applications.

show abstract