High-Dimensional Computing as a Nanoscalable Paradigm

Rahimi, Abbas; Datta, Sohum; Kleyko, Denis; Frady, E. Paxon; Olshausen, Bruno A.; Kanerva, Pentti; Rabaey, Jan M.

doi:10.1109/tcsi.2017.2705051

Cited by 124 publications

(86 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…is is equal to computing the population count of a hypervector binding those two hypervectors. So far, digital methods for AMs count through all components resulting in a classi cation latency in the order O(D) [8,9,29,32]. We focus on reducing this latency by adding up all hypervector components.…”

Section: Associative Memory (Am)mentioning

confidence: 99%

Hardware Optimizations of Dense Binary Hyperdimensional Computing: Rematerialization of Hypervectors, Binarized Bundling, and Combinational Associative Memory

Schmuck

Benini

Rahimi

2019

J. Emerg. Technol. Comput. Syst.

Self Cite

View full text Add to dashboard Cite

Brain-inspired hyperdimensional (HD) computing models neural activity pa erns of the very size of the brain's circuits with points of a hyperdimensional space, that is, with hypervectors. Hypervectors are D-dimensional (pseudo)random vectors with independent and identically distributed (i.i.d.) components constituting ultra-wide holographic words: D = 10, 000 bits, for instance. At its very core, HD computing manipulates a set of seed hypervectors to build composite hypervectors representing objects of interest. It demands memory optimizations with simple operations for an e cient hardware realization. In this paper, we propose hardware techniques for optimizations of HD computing, in a synthesizable open-source VHDL library, to enable co-located implementation of both learning and classi cation tasks on only a small portion of Xilinx ® UltraScale ™ FPGAs: (1) We propose simple logical operations to rematerialize the hypervectors on the y rather than loading them from memory. ese operations massively reduce the memory footprint by directly computing the composite hypervectors whose individual seed hypervectors do not need to be stored in memory.(2) Bundling a series of hypervectors over time requires a multibit counter per every hypervector component. We instead propose a binarized back-to-back bundling without requiring any counters. is truly enables on-chip learning with minimal resources as every hypervector component remains binary over the course of training to avoid otherwise multibit components. (3) For every classi cation event, an associative memory is in charge of nding the closest match between a set of learned hypervectors and a query hypervector by using a distance metric. is operator is proportional to hypervector dimension (D), and hence may take O(D) cycles per classi cation event. Accordingly, we signi cantly improve the throughput of classi cation by proposing associative memories that steadily reduce the latency of classi cation to the extreme of a single cycle. (4) We perform a design space exploration incorporating the proposed techniques on FPGAs for a wearable biosignal processing application as a case study. Our techniques achieve up to 2.39× area saving, or 2337× throughput improvement. e Pareto optimal HD architecture is mapped on only 18340 con gurable logic blocks (CLBs) to learn and classify ve hand gestures using four electromyography sensors.

show abstract

Section: Associative Memory (Am)mentioning

confidence: 99%

Hardware Optimizations of Dense Binary Hyperdimensional Computing: Rematerialization of Hypervectors, Binarized Bundling, and Combinational Associative Memory

Schmuck

Benini

Rahimi

2019

J. Emerg. Technol. Comput. Syst.

Self Cite

View full text Add to dashboard Cite

show abstract

“…This lets us combine two such vectors into a new vector using well-defined vector space operations, while keeping the information of the two vectors with high probability. HD computing has further unique features including fast learning, robustness, and efficiency of realization [10].…”

Section: B Hyperdimensional (Hd) Computingmentioning

confidence: 99%

One-shot Learning for iEEG Seizure Detection Using End-to-end Binary Operations: Local Binary Patterns with Hyperdimensional Computing

Burrello

Schindler

Benini

et al. 2018

2018 IEEE Biomedical Circuits and Systems Conference (BioCAS)

Self Cite

View full text Add to dashboard Cite

This paper presents an efficient binarized algorithm for both learning and classification of human epileptic seizures from intracranial electroencephalography (iEEG). The algorithm combines local binary patterns with brain-inspired hyperdimensional computing to enable end-to-end learning and inference with binary operations. The algorithm first transforms iEEG time series from each electrode into local binary pattern codes. Then atomic high-dimensional binary vectors are used to construct composite representations of seizures across all electrodes. For the majority of our patients (10 out of 16), the algorithm quickly learns from one or two seizures (i.e., one-/few-shot learning) and perfectly generalizes on 27 further seizures. For other patients, the algorithm requires three to six seizures for learning. Overall, our algorithm surpasses the state-of-the-art methods [1] for detecting 65 novel seizures with higher specificity and sensitivity, and lower memory footprint.

show abstract

“…The AM compares the query hypervectors to all learned prototype hypervectors, and returns the label of the one that has the minimum Hamming distance. Since these three modules are commonly used across various applications of HD computing [19][20][21], we target their accelerations to achieve end-to-end benefits in learning and classification tasks.…”

Section: Modules Of Hd Classifiermentioning

confidence: 99%

“…The mean classification accuracy of gestures among five subjects is 89.6% with SVM, and 92.4% with the HD classifier. More importantly, the HD classifier exhibits a graceful degradation with lower dimensionality, or faulty components, allowing a trade-off between the application's accuracy and the available hardware resources in a platform [19,20]. We can exploit this graceful degradation capability by reducing the dimensionality of hypervectors that eases the execution on the commercial ARM Cortex M4.…”

Section: Comparison With Svm On Arm Cortex M4mentioning

confidence: 99%

“…Such hypervectors can be mathematically manipulated to not only classify but also to make associations, form hierarchies, and perform other types of cognitive computations [9]. Key properties of HD computing include generality, scalability, a well-defined set of arithmetic operations on hypervectors, ubiquitous parallel operations, robustness, and graceful degradation making it possible to develop efficient nanoscalable learning machines [20].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

PULP-HD: Accelerating Brain-Inspired High-Dimensional Computing on a Parallel Ultra-Low Power Platform

Montagna

Rahimi

Benatti

et al. 2018

2018 55th ACM/ESDA/IEEE Design Automation Conference (DAC)

Self Cite

View full text Add to dashboard Cite

Computing with high-dimensional (HD) vectors, also referred to as hypervectors, is a brain-inspired alternative to computing with scalars. Key properties of HD computing include a well-defined set of arithmetic operations on hypervectors, generality, scalability, robustness, fast learning, and ubiquitous parallel operations. HD computing is about manipulating and comparing large patternsbinary hypervectors with 10,000 dimensions-making its efficient realization on minimalistic ultra-low-power platforms challenging. This paper describes HD computing's acceleration and its optimization of memory accesses and operations on a silicon prototype of the PULPv3 4-core platform (1.5 mm 2 , 2 mW), surpassing the stateof-the-art classification accuracy (on average 92.4%) with simultaneous 3.7× end-to-end speed-up and 2× energy saving compared to its single-core execution. We further explore the scalability of our accelerator by increasing the number of inputs and classification window on a new generation of the PULP architecture featuring bitmanipulation instruction extensions and larger number of 8 cores. These together enable a near ideal speed-up of 18.4× compared to the single-core PULPv3.

show abstract

High-Dimensional Computing as a Nanoscalable Paradigm

Cited by 124 publications

References 33 publications

Hardware Optimizations of Dense Binary Hyperdimensional Computing: Rematerialization of Hypervectors, Binarized Bundling, and Combinational Associative Memory

Hardware Optimizations of Dense Binary Hyperdimensional Computing: Rematerialization of Hypervectors, Binarized Bundling, and Combinational Associative Memory

One-shot Learning for iEEG Seizure Detection Using End-to-end Binary Operations: Local Binary Patterns with Hyperdimensional Computing

PULP-HD: Accelerating Brain-Inspired High-Dimensional Computing on a Parallel Ultra-Low Power Platform

Contact Info

Product

Resources

About