Deep Belief Network Training Improvement Using Elite Samples Minimizing Free Energy

Keyvanrad, Mohammad Ali; Homayounpour, Mohammad Mehdi

doi:10.1142/s0218001415510064

Cited by 21 publications

(6 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although all model parameters are changed in each step, PCD can receive good samples from model distribution with a few Gibbs sampling steps because the model parameters change slightly. As an improvement of PCD, FEPCD is based on Free Energy to generate better samples (Hinton, 2012; Keyvanrad and Homayounpour, 2015). The selection criteria for the optimal chain based on the free energy of visible layer sample is as followswhere F ( v ) is the free energy.…”

Section: Methodsmentioning

confidence: 99%

“…To overcome the shortcoming, a criterion for goodness of a chain called free energy in persistent contrastive divergence (FEPCD) can ensure the network model to obtain better chain selection in sampling learning, which improves the quality and efficiency of gradient approximation (Hinton, 2012). As a result, the approximation and classification ability of DBN model increases along with FEPCD (Keyvanrad and Homayounpour, 2015). However, the sampling methods of DBN used in the up-to-date bearing fault diagnosis applications are mainly traditional CD or PCD, which may lead to a gradual decline in learning ability of DBN in the long-term training process.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Fault recognition of large-size low-speed slewing bearing based on improved deep belief network

Pan

Wang

Chen

et al. 2022

Journal of Vibration and Control

View full text Add to dashboard Cite

Slewing bearing is one of critical transmission in wind turbine and shield machine withstanding low-speed and heavy-load working condition. Fault recognition is crucial to their high reliability operation. Many studies have been conducted using traditional shallow networks for fault recognition. However, they suffer from inherent disadvantages, such as low learning ability under high-dimensional nonlinear features, which make them unsuitable for fault recognition of slewing bearing. To solve these shortcomings, a novel fault recognition method is proposed based on improved deep belief network (DBN) using sampling method of free energy in persistent contrastive divergence (FEPCD). A systematic methodology based on multi-domain feature extraction is proposed to describe the fault characteristic information. After that, improved DBN optimized by FEPCD is employed to capture the fault features and recognize the fault condition of slewing bearing. The application and superiority of proposed methodology are validated using a slewing bearing life-cycle test dataset. Meanwhile, a comparison is conducted between traditional sampling methods contrastive divergence (CD) and persistent contrastive divergence (PCD). The results illustrate that improved FEPCD gets better result in training sampling. Compared with other deep learning methods such as deep Boltzmann machine (DBM) and stacked auto-encoder (SAE), and shallow intelligent algorithms like back propagation (BP) neural network and support vector machine (SVM), the fault recognition accuracy of slewing bearing is improved by using the improved DBN with FEPCD.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Fault recognition of large-size low-speed slewing bearing based on improved deep belief network

Pan

Wang

Chen

et al. 2022

Journal of Vibration and Control

View full text Add to dashboard Cite

show abstract

“…Once RBM is learned using Contrastive Divergence (CD) algorithm [19], the DBN is able to initialize the weights of feed forward back-propagation neural network then it is used for classification to predict the image model. RBM can be learnt better when the predictive model is used before the step sampling in Gibs before collecting statistical step in learning rule but for the purposes of pre-training.…”

Section: Contrastive Divergencementioning

confidence: 99%

“…It has produced the state-of-the-art results on recognition and classification tasks [10]. On the other hand, typical classification methods used for speech recognition include hidden Markov model (HMM) [14], Gaussian Mixture Model (GMM) [15], artificial neural networks such as recurrent neural network (RNN) [16], support vector machine (SVM) [17,18], and the fuzzy cognitive map network [19]. These methods are confronted with the complicated decision boundary of the classification.…”

Section: Introductionmentioning

confidence: 99%

Enhancing Single Speaker Recognition Using Deep Belief Network

Prasetio

Hayashida

Nishizaki

et al. 2018

TMLAI

View full text Add to dashboard Cite

Recognition in speech is complex phenomena study, and the reason for this is the complexity of human language. The barrier of the problem in speech recognition study now can be handled from speech signal using machine learning methods. Nowadays, Deep Belief Networks (DBN) automatically is able to find out the representation of speech signal. This paper tries to approach a structure optimization of DBN which based on the combined technique of evolutionary computation to enhance the single speaker speech. It firstly extracts from the feature of speech signal then applies them to construct lots of random subspaces. The result of the conducted experimental in the evolutionary computation of DBN indicates the structure have an improvement for speech recognition.

show abstract

“…Whereas CD has some disadvantages and is not exact, other methods are proposed in REM. One of these methods is PCD that is very popular [13] and another method is FEPCD that has been proposed by authors in [14].…”

Section: Deep Belief Network (Dbns) and Restricted Boltzmann Macmentioning

confidence: 99%

Normal sparse Deep Belief Network

Keyvanrad

Homayounpour

2015

2015 International Joint Conference on Neural Networks (IJCNN)

Self Cite

View full text Add to dashboard Cite

Nowadays this is very popular to use deep architectures in machine learning. Deep Belief Networks (DBNs) have deep architectures to create a powerful generative model using training data. Deep Belief Networks can be used in classification and feature learning. A DBN can be learnt unsupervised and then the learnt features are suitable for a simple classifier (like a linear classifier) with a few labeled data. According to researches, training of DBN can be improved to produce features with more interpretability and discrimination ability. One of these improvements is sparsity in learnt features in DBN. By using sparsity we can learn useful low-level feature representations for unlabeled data. In sparse representation we benefit from this property that the learnt features can be interpreted, i.e. they correspond to meaningful aspects of the input, and capture factors of variation in the data. Different methods have been proposed to build sparse RBMs. In this paper we propose a new method namely nsDBN that has different behaviors according to deviation of the activation of the hidden units from a (low) fixed value. Also our proposed method has a variance parameter that can control the force degree of sparseness. According to the results, our new method compared to the state of the art methods including peA, RBM, qsRBM, and rdsRBM always achieves the best recognition accuracy on the MNIST hand written digit recognition test set even when only 10 to 20 labeled samples per class are used as training data.

show abstract

Deep Belief Network Training Improvement Using Elite Samples Minimizing Free Energy

Cited by 21 publications

References 14 publications

Fault recognition of large-size low-speed slewing bearing based on improved deep belief network

Fault recognition of large-size low-speed slewing bearing based on improved deep belief network

Enhancing Single Speaker Recognition Using Deep Belief Network

Normal sparse Deep Belief Network

Contact Info

Product

Resources

About