Accelerating Training of Deep Neural Networks on GPU using CUDA

The Boltzmann machine (BM) and restricted Boltzmann machine (RBM) models are representative stochastic neural networks, in which neuron states are determined by stochastic activation functions. They are widely used in many applications. However, when analog circuits are used to realize a neural network, noise are not avoidable. The noise could come from external environments, such as power supplies and thermal noise, and they will affect the neurons' stochastic behaviour in BM and RBM. Hence it is important to theoretically study how the noise affect the BM and RBM. To best of our knowledge, there are little works related to the analysis on noisy BMs and noisy RBMs. This paper considers that there are additive noise in the neurons of BMs or RBMs, and theoretically studies the behaviors of BM and RBM under this imperfect condition. It is found that the effect of additive noise is similar to increasing the temperature factor of BM and RBM. Since the input noise may make the networks to have wrong stochastic behaviour, there is Kullback Leibler (KL) divergence loss in noisy BMs and noisy RBMs. Based on the Gaussian-distributed noise assumption, a noise compensation method is also proposed to suppress the effect of additive noise. Experiments show that the proposed noise compensation method can greatly suppress the KL divergence loss. In addition, from the experimental results, our method is also effective for handling non-Gaussian noise.

show abstract

“…By averaging, we get the measured probability of turning on for each given u. The approximation probability is derived from (19). Fig.…”

Section: A Approximation Ability Of Lemmamentioning

confidence: 99%

Analysis on Noisy Boltzmann Machines and Noisy Restricted Boltzmann Machines

Leung

Sum

2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…and GPUs [6], [7]. Due to the demand for an accelerated high computational environment, an algorithm is required to decrease the execution time and improves performance [8]. Rather than doing shifting and allocating the memory to the host and device allocate a special pointer that can be used by both CPU and GPU, this is the concept of unified memory allocation [9].…”

mentioning

confidence: 99%

Heterogeneous computing with graphical processing unit: improvised back-propagation algorithm for water level prediction

Singh

Panda

2022

IJECE

View full text Add to dashboard Cite

<p>A multitude of research has been rising for predicting the behavior of different real-world problems through machine learning models. An erratic nature occurs due to the augmented behavior and inadequacy of the prerequisite dataset for the prediction of water level over different fundamental models that show flat or low-set accuracy. In this paper, a powerful scaling strategy is proposed for improvised back-propagation algorithm using parallel computing for groundwater level prediction on graphical processing unit (GPU) for the Faridabad region, Haryana, India. This paper aims to propose the new streamlined form of a back-propagation algorithm for heterogeneous computing and to examine the coalescence of artificial neural network (ANN) with GPU for predicting the groundwater level. twenty years of data set from 2001-2020 has been taken into consideration for three input parameters namely, temperature, rainfall, and water level for predicting the groundwater level using parallelized backpropagation algorithm on compute unified device architecture (CUDA). This employs the back-propagation algorithm to be best suited to reinforce learning and performance by providing more accurate and fast results for water level predictions on GPUs as compared to sequential ones on central processing units (CPUs) alone.</p>

show abstract