Trained Neural Networks Ensembles Weight Connections Analysis

Ahmed, Muhammad Atta Othman

doi:10.1007/978-3-319-74690-6_24

Cited by 10 publications

(5 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is interesting in itself, but will hopefully also open up novel pathways for the mathematical analysis of the behaviour of spiking neural networks trained using Hebbian learning. In particular, it may lead to novel insights about the statistical distribution of weights, which has been of substantial research interest [28,29,30,31]. Secondly and more practically relevant, as we discuss below, our results can be used to adapt the learning rate of the neuron.…”

Section: Introductionmentioning

confidence: 81%

Constraints on Hebbian and STDP learned weights of a spiking neuron

Chu

Nguyêݱn

2021

Neural Networks

View full text Add to dashboard Cite

show abstract

Section: Introductionmentioning

confidence: 81%

Constraints on Hebbian and STDP learned weights of a spiking neuron

Chu

Nguyêݱn

2021

Neural Networks

View full text Add to dashboard Cite

show abstract

“…Suppose the two output neurons are y (0) and y (1) , and assume y (0) > y (1) without loss of generality. When there are random errors injected to the neural network, the two output neurons follow normal distribution accordingly and they can be formulated with Y (0) ∼ N (y (0) , var(∆ y ), Y (1) ∼ N (y (1) , var(∆ y )) according to Section III-B. In this case, the probability of wrong classification is essentially that of Y (0) < Y (1) .…”

Section: Relation Between Rrmse and Classification Accuracymentioning

confidence: 99%

“…When there are random errors injected to the neural network, the two output neurons follow normal distribution accordingly and they can be formulated with Y (0) ∼ N (y (0) , var(∆ y ), Y (1) ∼ N (y (1) , var(∆ y )) according to Section III-B. In this case, the probability of wrong classification is essentially that of Y (0) < Y (1) . The distribution of Y (0) < Y (1) can be formulated with Equation 13, which is a shifted and scaled normal distribution model.…”

Section: Relation Between Rrmse and Classification Accuracymentioning

confidence: 99%

“…In this case, the probability of wrong classification is essentially that of Y (0) < Y (1) . The distribution of Y (0) < Y (1) can be formulated with Equation 13, which is a shifted and scaled normal distribution model. Hence, the probability when Y (0) − Y (1) > 0 can be calculated with Equation 14where normcdf stands for cumulative distribution function of normal distribution.…”

Section: Relation Between Rrmse and Classification Accuracymentioning

confidence: 99%

See 1 more Smart Citation

Statistical Modeling of Soft Error Influence on Neural Networks

Huang¹,

Xue²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Soft errors in large VLSI circuits pose dramatic influence on computing-and memory-intensive neural network (NN) processing. Understanding the influence of soft errors on NNs is critical to protect against soft errors for reliable NN processing. Prior work mainly rely on fault simulation to analyze the influence of soft errors on NN processing. They are accurate but usually specific to limited configurations of errors and NN models due to the prohibitively slow simulation speed especially for large NN models and datasets. With the observation that the influence of soft errors propagates across a large number of neurons and accumulates as well, we propose to characterize the soft error induced data disturbance on each neuron with normal distribution model according to central limit theorem and develop a series of statistical models to analyze the behavior of NN models under soft errors in general. The statistical models reveal not only the correlation between soft errors and NN model accuracy, but also how NN parameters such as quantization and architecture affect the reliability of NNs. The proposed models are compared with fault simulation and verified comprehensively. In addition, we observe that the statistical models that characterize the soft error influence can also be utilized to predict fault simulation results in many cases and we explore the use of the proposed statistical models to accelerate fault simulations of NNs. According to our experiments, the accelerated fault simulation shows almost two orders of magnitude speedup with negligible simulation accuracy loss over the baseline fault simulations.

show abstract

“…But normal and uniform distribution are commonly used initialization weight distributions [16,19,20]. Meanwhile, [1] shows that many weights may fit the t location-scale distribution. When w follows the above distribution, h and g bring strong correlation with the constraint µ |x| = µ |w| .…”

Section: Similar Operationsmentioning

confidence: 99%

MinConvNets: A new class of multiplication-less Neural Networks

Yang¹,

Chaudhuri²,

Likforman³

et al. 2021

Preprint

View full text Add to dashboard Cite

Convolutional Neural Networks have achieved unprecedented success in image classification, recognition, or detection applications. However, their large-scale deployment in embedded devices is still limited by the huge computational requirements, i.e., millions of MAC operations per layer. In this article, MinConvNets where the multiplications in the forward propagation are approximated by minimum comparator operations are introduced. Hardware implementation of minimum operation is much simpler than multipliers. Firstly, a methodology to find approximate operations based on statistical correlation is presented. We show that it is possible to replace multipliers by minimum operations in the forward propagation under certain constraints, i.e. given similar mean and variances of the feature and the weight vectors. A modified training method which guarantees the above constraints is proposed. And it is shown that equivalent precision can be achieved during inference with MinConvNets by using transfer learning from well trained exact CNNs.

show abstract

Trained Neural Networks Ensembles Weight Connections Analysis

Cited by 10 publications

References 17 publications

Constraints on Hebbian and STDP learned weights of a spiking neuron

Constraints on Hebbian and STDP learned weights of a spiking neuron

Statistical Modeling of Soft Error Influence on Neural Networks

MinConvNets: A new class of multiplication-less Neural Networks

Contact Info

Product

Resources

About