A topological insight into restricted Boltzmann machines

Abstract

Restricted Boltzmann Machines (RBMs) and models derived from them have been successfully used as basic building blocks in deep artificial neural networks for automatic features extraction, unsupervised weights initialization, but also as density estimators. Thus, their generative and discriminative capabilities, but also their computational time are instrumental to a wide range of applications. Our main contribution is to look at RBMs from a topological perspective, bringing insights from network science. Firstly, here we show that RBMs and Gaussian RBMs (GRBMs) are bipartite graphs which naturally have a small-world topology. Secondly, we demonstrate both on synthetic and real-world datasets that by constraining RBMs and GRBMs to a scale-free topology (while still considering local neighborhoods and data distribution), we reduce the number of weights that need to be computed by a few orders of magnitude, at virtually no loss in generative performance. Thirdly, we show that, for a fixed number of weights, our proposed sparse models (which by design have a higher number of hidden neurons) achieve better generative capabilities than standard fully connected RBMs and GRBMs (which by design have a smaller number of hidden neurons), at no additional computational costs.Comment: http://link.springer.com/article/10.1007/s10994-016-5570-z, Machine Learning, issn=1573-0565, 201

Learnability scaling of quantum states: Restricted Boltzmann machines

“…Following the ideas of Refs. [31,32], we define a pruning procedure for our scaling study in the following steps:…”

Comparison of Deep Learning Algorithms to Predict Customer Churn within a Local Retail Industry

“…Restricted Boltzmann Machine (RBM)are a stochastic neural network, whereby the algorithm is used to identify patterns within data [11]. A RBM consists of three layers, the visible layer, hidden layer and output layer.…”

“…The former focuses on improving the likelihood function for a joint distribution, whilst the second is a conditional distribution. The ClassRBM, may be used as a standalone classifier or in conjunction with other models [11].…”

Unsupervised deep learning for real-time assessment of video streaming services

“…Second, to master the sheer scale of the problem, we selected Deep Learning (DL) techniques. Within this type of techniques, the Restricted Boltzmann Machines (RBMs) have demonstrated outstanding performance as density estimators [25]. This characteristic made us chose them for still images quality estimation [22,24].…”