A Scalable and Programmable Architecture for the Continuous Restricted Boltzmann Machine in VLSI

An embedded probabilistic neural network with on-chip learning capability

“…Finally, for a j x ≥1 and a j x ≤-1, φ(x) is set to be +1 and -1, respectively. Fig.2a shows the difference between the ideal sigmoid function and its PWL approximation according to (5). The simulation results in Fig.2b and Fig.2c indicate that, under this approximation, the CRBM is able to model three types of eNose data, such that different types of data result in significantly different responses of hidden neurons.…”

“…If each sigmoid curve is approximated by a PWL curve with only three segments (ϕ(x)=1, 0.5a j x, and -1), our pilot simulation indicates that the approximation error is too large such that the CRBM is not able to learn data distributions successfully. Therefore, the sigmoid function is approximated by a PWL function with five segments, as given in (5). φ(x) is approximated as 0.5a j x only when |a j x|<0.5.…”

“…According to [5], the learning rule in (3) requires 12-bit resolution for adapting all parameters to their optimal values. To incorporate on-chip learning ability in the dCRBM system, the multiplier is designed to take 12-bit inputs and to generate a 24-bit output.…”

“…A probabilistic neural network called the Continuous Restricted Boltzmann Machine (CRBM) has been shown capable of classifying biomedical data reliably [4]. Analog VLSI implementation of the CRBM has also been demonstrated in [5]. However, as the advanced technology continues to shrink the transistor size, transistor mismatches and nonlinearity grow so significantly [6] that the CRBM system requires complex analog circuits to achieve enough (12-bit) resolution for parameter training [5].…”

“…Analog VLSI implementation of the CRBM has also been demonstrated in [5]. However, as the advanced technology continues to shrink the transistor size, transistor mismatches and nonlinearity grow so significantly [6] that the CRBM system requires complex analog circuits to achieve enough (12-bit) resolution for parameter training [5]. The complexity makes analog design lose the power and area advantages over digital design.…”