Vector Symbolic Architectures as a Computing Framework for Nanoscale Hardware

Kleyko, Denis; Davies, Martin; Frady, E. Paxon; Kanerva, Pentti; Kent, Spencer J.; Olshausen, Bruno A.; Osipov, Evgeny; Rabaey, Jan M.; Rachkovskij, Dmitri A.; Rahimi, Abbas; Sommer, Friedrich T.

doi:10.48550/arxiv.2106.05268

Cited by 10 publications

(15 citation statements)

References 69 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this work, we enhance the machine learning method by exploring and translating the memory processing capability of the brain [40]. To maximize the synergy between anthropogenic concepts and a body in silico, we analyzed the distinct neuromorphic nature of SNN and Vector Symbolic Architecture (VSA) [44,45]. We found that the two studies approach neuromorphic computing from complementary philosophies: SNN embodies the sensory processing patterns of the brain from a biological standpoint, while the VSA approach processes data from the behavioral patterns.…”

Section: Analogy From the Brainmentioning

confidence: 99%

Spiking Hyperdimensional Network: Neuromorphic Models Integrated with Memory-Inspired Framework

Zou,

Alimohamadi,

Imani

et al. 2021

Preprint

View full text Add to dashboard Cite

Recently, brain-inspired computing models have shown great potential to outperform today's deep learning solutions in terms of robustness and energy efficiency. Particularly, Spiking Neural Networks (SNNs) and HyperDimensional Computing (HDC) have shown promising results in enabling efficient and robust cognitive learning. Despite the success, these two brain-inspired models have different strengths. While SNN mimics the physical properties of the human brain, HDC models the brain on a more abstract and functional level. Their design philosophies demonstrate complementary patterns that motivate their combination. With the help of the classical psychological model on memory, we propose SpikeHD, the first framework that fundamentally combines Spiking neural network and hyperdimensional computing. SpikeHD generates a scalable and strong cognitive learning system that better mimics brain functionality. SpikeHD exploits spiking neural networks to extract low-level features by preserving the spatial and temporal correlation of raw event-based spike data. Then, it utilizes HDC to operate over SNN output by mapping the signal into high-dimensional space, learning the abstract information, and classifying the data. Our extensive evaluation on a set of benchmark classification problems shows that SpikeHD provides the following benefit compared to SNN architecture:(1) significantly enhance learning capability by exploiting two-stage information processing, (2) enables substantial robustness to noise and failure, and (3) reduces the network size and required parameters to learn complex information.

show abstract

Section: Analogy From the Brainmentioning

confidence: 99%

Spiking Hyperdimensional Network: Neuromorphic Models Integrated with Memory-Inspired Framework

Zou,

Alimohamadi,

Imani

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Examples of the new paradigms are neuromorphic and nanoscalable computing, where HDC/VSA is prospected to play an important role (see [Kleyko et al, 2021a] and references therein for perspective). Due to this surge of interest in HDC/VSA, the need for providing the broad coverage of the area, which is currently missing, became evident.…”

Section: Introductionmentioning

confidence: 99%

“…cognitive architectures [Plate, 1997a] ± ± [Kanerva, 2009] ± ± ± ± ± [Neubert et al, 2019] ± ± ± ± [Ge and Parhi, 2020] [ Schlegel et al, 2020] ± ± ± [Kleyko et al, 2021a] ± [Hassan et al, 2021…”

mentioning

confidence: 99%

A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges

Kleyko,

Rachkovskij,

Osipov

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

This is Part II of the two-part comprehensive survey devoted to a computing framework most commonly known under the names Hyperdimensional Computing and Vector Symbolic Architectures (HDC/VSA). Both names refer to a family of computational models that use high-dimensional distributed representations and rely on the algebraic properties of their key operations to incorporate the advantages of structured symbolic representations and vector distributed representations. Holographic Reduced Representations [Plate, 1995], [Plate, 2003] is an influential HDC/VSA model that is well-known in the machine learning domain and often used to refer to the whole family. However, for the sake of consistency, we use HDC/VSA to refer to the area.Part I of this survey [Kleyko et al., 2021c] covered foundational aspects of the area, such as historical context leading to the development of HDC/VSA, key elements of any HDC/VSA model, known HDC/VSA models, and transforming input data of various types into high-dimensional vectors suitable for HDC/VSA. This second part surveys existing applications, the role of HDC/VSA in cognitive computing and architectures, as well as directions for future work. Most of the applications lie within the machine learning/artificial intelligence domain, however we also cover other applications to provide a thorough picture. The survey is written to be useful for both newcomers and practitioners.

show abstract

“…In classification tasks, the use of VSA leads to order of magnitude increase in energy efficiency of computations on the one hand and natively enables oneshot and multitask learning on the other [7]. It is prospected that VSA will play a key role in the development of novel neuromorphic computer architectures [8] as an algorithmic abstraction [9], [10]. The main contribution of this paper is a novel algorithm for unsupervised learning called Hyperseed, which relies on the mathematical properties of arising from random representation spaces described through the concentration of measure [11] and of the main VSA operations of binding and bundling [12].…”

Section: Introductionmentioning

confidence: 99%

HyperSeed: Unsupervised Learning with Vector Symbolic Architectures

Osipov¹,

Kahawala²,

Haputhanthri³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Motivated by recent innovations in biologicallyinspired neuromorphic hardware, this paper presents a novel unsupervised machine learning approach named Hyperseed that leverages Vector Symbolic Architectures (VSA) for fast learning a topology preserving feature map of unlabelled data. It relies on two major capabilities of VSAs: the binding operation and computing in superposition. In this paper, we introduce the algorithmic part of Hyperseed expressed within Fourier Holographic Reduced Representations VSA model, which is specifically suited for implementation on spiking neuromorphic hardware. The two distinctive novelties of the Hyperseed algorithm are: 1) Learning from only few input data samples and 2) A learning rule based on a single vector operation. These properties are demonstrated on synthetic datasets as well as on illustrative benchmark usecases, IRIS classification and a language identification task using n-gram statistics.

show abstract

Vector Symbolic Architectures as a Computing Framework for Nanoscale Hardware

Cited by 10 publications

References 69 publications

Spiking Hyperdimensional Network: Neuromorphic Models Integrated with Memory-Inspired Framework

Spiking Hyperdimensional Network: Neuromorphic Models Integrated with Memory-Inspired Framework

A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges

HyperSeed: Unsupervised Learning with Vector Symbolic Architectures

Contact Info

Product

Resources

About