Hierarchical temporal memory implementation on FPGA using LFSR based spatial pooler address space generator

Kerner, Madis; Tammemäe, Kalle

doi:10.1109/ddecs.2017.7934577

Cited by 4 publications

(6 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then, the same prediction is made using the HTM-HW model. 14 Fig. 10a shows the accumulated MAPE recorded at every 250 samples.…”

Section: Time-series Predictionmentioning

confidence: 99%

“…To this end, several research groups have attempted to develop specialized custom hardware designs to run the HTM algorithm efficiently and affordably [12]. While some of the previous designs focused only on the spatial aspect of the HTM [13], [14], [15], other endeavors incorporated both the spatial and temporal models in the same design. For instance, in 2015, Zyarah et al implemented the HTM algorithm including the spatial and temporal aspects [16].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Neuromorphic System for Spatial and Temporal Information Processing

Zyarah

Gómez

Kudithipudi

2020

IEEE Trans. Comput.

View full text Add to dashboard Cite

Neuromorphic systems that learn and predict from streaming inputs hold significant promise in pervasive edge computing and its applications. In this article, a neuromorphic system that processes spatio-temporal information on the edge is proposed. Algorithmically, the system is based on hierarchical temporal memory that inherently offers online learning, resiliency, and fault tolerance. Architecturally, it is a full custom mixed-signal design with an underlying digital communication scheme and analog computational modules. Therefore, the proposed system features reconfigurability, real-time processing, low power consumption, and low-latency processing. The proposed architecture is benchmarked to predict on real-world streaming data. The network's mean absolute percentage error on the mixed-signal system is 1.129 X lower compared to its baseline algorithm model. This reduction can be attributed to device non-idealities and probabilistic formation of synaptic connections. We demonstrate that the combined effect of Hebbian learning and network sparsity also plays a major role in extending the overall network lifespan. We also illustrate that the system offers 3.46 X reduction in latency and 77.02 X reduction in power consumption when compared to a custom CMOS digital design implemented at the same technology node. By employing specific low power techniques, such as clock gating, we observe 161.37 X reduction in power consumption.

show abstract

“…Then, the same prediction is made using the HTM-HW model. 14 Fig. 10a shows the accumulated MAPE recorded at every 250 samples.…”

Section: Time-series Predictionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Neuromorphic System for Spatial and Temporal Information Processing

Zyarah

Gómez

Kudithipudi

2020

IEEE Trans. Comput.

View full text Add to dashboard Cite

show abstract

“…Initially, the golden software model, HTM-SW, is used in the prediction. Then, the same prediction is made using the HTM-HW model 14 . Fig.…”

Section: Time-series Predictionmentioning

confidence: 99%

“…10-(a) shows the accumulated MAPE recorded at every 250 samples. It can be seen that the initial value of the MAPE is really high, but over time it decreases as the network learns patterns and uses the acquired knowledge to make valid predictions 14 HTM-HW model is also benchmarked using other datasets such as NYC-Taxi [42]. The achieved MAPE for the 2nd and 5th order predictions are 0.0996±0.0014 and 0.156 ± 0.0084, respectively.…”

Section: Time-series Predictionmentioning

confidence: 99%

End-to-End Memristive HTM System for Pattern Recognition and Sequence Prediction

Zyarah,

Gomez,

Kudithipudi

2020

Preprint

View full text Add to dashboard Cite

Neuromorphic systems that learn and predict from streaming inputs hold significant promise in pervasive edge computing and its applications. In this paper, a neuromorphic system that processes spatio-temporal information on the edge is proposed. Algorithmically, the system is based on hierarchical temporal memory that inherently offers online learning, resiliency, and fault tolerance. Architecturally, it is a full custom mixed-signal design with an underlying digital communication scheme and analog computational modules. Therefore, the proposed system features reconfigurability, real-time processing, low power consumption, and low-latency processing. The proposed architecture is benchmarked to predict on real-world streaming data. The network's mean absolute percentage error on the mixed-signal system is 1.129X lower compared to its baseline algorithm model. This reduction can be attributed to device non-idealities and probabilistic formation of synaptic connections. We demonstrate that the combined effect of Hebbian learning and network sparsity also plays a major role in extending the overall network lifespan. We also illustrate that the system offers 3.46X reduction in latency and 77.02X reduction in power consumption when compared to a custom CMOS digital design implemented at the same technology node. By employing specific low power techniques, such as clock gating, we observe 161.37X reduction in power consumption.

show abstract

“…Non-volatile HTM [12] Memrsitive HTM [13] LFSR HTM [25] Digital HTM [14] This work (PIM HTM) Spatial Spatial Spatial Spatial-temporal Spatial-Temporal a In the Digital HTM [14], the power of the register files is not included. b In these references, only the power of the SP is reported.…”

Section: Algorithmmentioning

confidence: 99%

Neuromorphic Architecture for the Hierarchical Temporal Memory

Zyarah

Kudithipudi

2019

IEEE Trans. Emerg. Top. Comput. Intell.

View full text Add to dashboard Cite

A biomimetic machine intelligence algorithm, that holds promise in creating invariant representations of spatiotemporal input streams is the hierarchical temporal memory (HTM). This unsupervised online algorithm has been demonstrated on several machine-learning tasks, including anomaly detection. Significant effort has been made in formalizing and applying the HTM algorithm to different classes of problems. There are few early explorations of the HTM hardware architecture, especially for the earlier version of the spatial pooler of HTM algorithm. In this article, we present a full-scale HTM architecture for both spatial pooler and temporal memory. Synthetic synapse design is proposed to address the potential and dynamic interconnections occurring during learning. The architecture is interweaved with parallel cells and columns that enable high processing speed for the HTM. The proposed architecture is verified for two different datasets: MNIST and the European number plate font (EUNF), with and without the presence of noise. The spatial pooler architecture is synthesized on Xilinx ZYNQ-7, with 91.16% classification accuracy for MNIST and 90% accuracy for EUNF, with noise. For the temporal memory sequence prediction, first and second order predictions are observed for a 5-number long sequence generated from EUNF dataset and 95% accuracy is obtained. Moreover, the proposed hardware architecture offers 1364X speedup over the software realization. These results indicate that the proposed architecture can serve as a digital core to build the HTM in hardware and eventually as a standalone self-learning system.

show abstract

Hierarchical temporal memory implementation on FPGA using LFSR based spatial pooler address space generator

Cited by 4 publications

References 2 publications

Neuromorphic System for Spatial and Temporal Information Processing

Neuromorphic System for Spatial and Temporal Information Processing

End-to-End Memristive HTM System for Pattern Recognition and Sequence Prediction

Neuromorphic Architecture for the Hierarchical Temporal Memory

Contact Info

Product

Resources

About