Formal Analysis of HTM Spatial Pooler Performance Under Predefined Operation Conditions

Pietroń, Marcin; Wielgosz, Maciej; Wiatr, K.

doi:10.1007/978-3-319-47160-0_36

Cited by 9 publications

(7 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Originally described in Hawkins et al ( 2011 ), the term “spatial pooler” is used because input patterns that share a large number of co-active neurons (i.e., that are spatially similar) are grouped together into a common output representation. Recently there has been increasing interest in the mathematical properties of the HTM spatial pooler (Pietron et al, 2016 ; Mnatzaganian et al, 2017 ) and machine learning applications based on it (Thornton and Srbic, 2013 ; Ibrayev et al, 2016 ). In this paper, we explore several functional properties of the HTM spatial pooler that have not yet been systematically analyzed.…”

Section: Introductionmentioning

confidence: 99%

The HTM Spatial Pooler—A Neocortical Algorithm for Online Sparse Distributed Coding

Cui

Ahmad

Hawkins

2017

Front. Comput. Neurosci.

View full text Add to dashboard Cite

Hierarchical temporal memory (HTM) provides a theoretical framework that models several key computational principles of the neocortex. In this paper, we analyze an important component of HTM, the HTM spatial pooler (SP). The SP models how neurons learn feedforward connections and form efficient representations of the input. It converts arbitrary binary input patterns into sparse distributed representations (SDRs) using a combination of competitive Hebbian learning rules and homeostatic excitability control. We describe a number of key properties of the SP, including fast adaptation to changing input statistics, improved noise robustness through learning, efficient use of cells, and robustness to cell death. In order to quantify these properties we develop a set of metrics that can be directly computed from the SP outputs. We show how the properties are met using these metrics and targeted artificial simulations. We then demonstrate the value of the SP in a complete end-to-end real-world HTM system. We discuss the relationship with neuroscience and previous studies of sparse coding. The HTM spatial pooler represents a neurally inspired algorithm for learning sparse representations from noisy data streams in an online fashion.

show abstract

Section: Introductionmentioning

confidence: 99%

The HTM Spatial Pooler—A Neocortical Algorithm for Online Sparse Distributed Coding

Cui

Ahmad

Hawkins

2017

Front. Comput. Neurosci.

View full text Add to dashboard Cite

show abstract

“…that are spatially similar) are grouped together into a common output representation. Recently there has been increasing interest in the mathematical properties of the HTM spatial pooler (Pietroń et al, 2016;Mnatzaganian et al, 2017) and machine learning applications based on it (Thornton and Srbic, 2011;Ibrayev et al, 2016). In this paper we explore several functional properties of the HTM spatial pooler that have not yet been systematically analyzed.…”

Section: Introductionmentioning

confidence: 99%

“…HTM spatial pooler is an important component of HTM and was originally described in the Numenta whitepaper (Hawkins et al, 2011). Recently there has been an increasing interest in the mathematical properties of the HTM spatial pooler (Mnatzaganian et al, 2016;Pietroń et al, 2016) and machine learning applications based on it (Thornton and Srbic, 2011;Ibrayev et al, 2016). However, the computational properties and design principles of HTM spatial pooler have not been well documented.…”

Section: Introductionmentioning

confidence: 99%

The HTM Spatial Pooler – a neocortical algorithm for online sparse distributed coding

Cui

Ahmad

Hawkins

2016

Preprint

View full text Add to dashboard Cite

1.AbstractHierarchical temporal memory (HTM) provides a theoretical framework that models several key computational principles of the neocortex. In this paper we analyze an important component of HTM, the HTM spatial pooler (SP). The SP models how neurons learn feedforward connections and form efficient representations of the input. It converts arbitrary binary input patterns into sparse distributed representations (SDRs) using a combination of competitive Hebbian learning rules and homeostatic excitability control. We describe a number of key properties of the spatial pooler, including fast adaptation to changing input statistics, improved noise robustness through learning, efficient use of cells and robustness to cell death. In order to quantify these properties we develop a set of metrics that can be directly computed from the spatial pooler outputs. We show how the properties are met using these metrics and targeted artificial simulations. We then demonstrate the value of the spatial pooler in a complete end-to-end real-world HTM system. We discuss the relationship with neuroscience and previous studies of sparse coding. The HTM spatial pooler represents a neurally inspired algorithm for learning sparse representations from noisy data streams in an online fashion.

show abstract

“…By doing so we aim to develop a complete system [12] working on the principles of the human brain as they were presented in [7,10] with our modification making the algorithm suitable for hardware implementation. Running HTM on CPU is very slow, and due to its strongly parallel structure the algorithm is a good candidate for General-Purpose Graphics Processing Unit (GPGPU) and Field-Programmable Gate Array (FPGA) acceleration [13,14]. Therefore, the computationally demanding overlap and inhibition sections of SP were implemented on GPU.…”

Section: Closer Examination Of Species' Evolution On Earth Showmentioning

confidence: 99%

Using Spatial Pooler of Hierarchical Temporal Memory to classify noisy videos with predefined complexity

Wielgosz

Pietro

2017

Neurocomputing

Self Cite

View full text Add to dashboard Cite

This paper examines the performance of a Spatial Pooler (SP) of a Hierarchical Temporal Memory (HTM) in the task of noisy object recognition. To address this challenge, a dedicated custom-designed system based on the SP, histogram calculation module and SVM classifier was implemented. In addition to implementing their own version of HTM, the authors also designed a profiler which is capable of tracing all of the key parameters of the system. This was necessary, since an analysis and monitoring of the system performance turned out to be extremely difficult using conventional testing and debugging tools.The system was initially trained on artificially prepared videos without noise and then tested with a set of noisy video streams. This approach was intended to mimic a real life scenario where an agent or a system trained to deal with ideal objects faces a task of classifying distorted and noisy ones in its regular working conditions.The authors conducted a series of experiments for various macro parameters of HTM SP, as well as for different levels of video reduction ratios. The experiments allowed them to evaluate the performance of two different system setups (i.e. 'Multiple HTMs' and 'Single HTM') under various noise conditions with 32-frame video files. Results of all the tests were compared to SVM baseline setup.It was determined that the system featuring SP is capable of achieving approximately 12 times the noise reduction for a video signal with with distorted bits accounting for 13% of the total. Furthermore, the system featuring SP performed better also in the experiments without a noise component and achieved a max F1 score of 0.96. The experiments also revealed that a rise of column and synapse number of SP has a substantial impact on the performance of the system. Consequently, the highest F1 score values were obtained for 256 and 4096 synapses and columns respectively. It is worth noting that 'Multiple HTMs' setup outperforms the 'Single HTM' one. Nevertheless, this is achieved at the expense of increased computational demands.

show abstract

Formal Analysis of HTM Spatial Pooler Performance Under Predefined Operation Conditions

Cited by 9 publications

References 12 publications

The HTM Spatial Pooler—A Neocortical Algorithm for Online Sparse Distributed Coding

The HTM Spatial Pooler—A Neocortical Algorithm for Online Sparse Distributed Coding

The HTM Spatial Pooler – a neocortical algorithm for online sparse distributed coding

Using Spatial Pooler of Hierarchical Temporal Memory to classify noisy videos with predefined complexity

Contact Info

Product

Resources

About