A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part I: Models and Data Transformations

Kleyko, Denis; Rachkovskij, Dmitri A.; Osipov, Evgeny; Rahimi, Abbas

doi:10.48550/arxiv.2111.06077

Cited by 5 publications

(41 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, a detailed account for the related work on representation of sequences was provided in [12], [38], so here we only sketch a grand schema. Largely, the approaches for transforming sequences into HVs are based on either multiplicative or permutative binding [44].…”

Section: A Related Workmentioning

confidence: 99%

“…Our methods are based on the recursive application of a binding operation to represent the order of sequence elements and were developed for the HDC/VSA model of Fourier Holographic Reduced Representations (FHRR) [27]. The proposed methods can be adapted for other HDC/VSA models that use multiplicative and recursive binding operation (see [6], [12]).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Recursive Binding for Similarity-Preserving Hypervector Representations of Sequences

Rachkovskij¹,

Kleyko²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Hyperdimensional computing (HDC), also known as vector symbolic architectures (VSA), is a computing framework used within artificial intelligence and cognitive computing that operates with distributed vector representations of large fixed dimensionality. A critical step for designing the HDC/VSA solutions is to obtain such representations from the input data. Here, we focus on sequences and propose their transformation to distributed representations that both preserve the similarity of identical sequence elements at nearby positions and are equivariant to the sequence shift. These properties are enabled by forming representations of sequence positions using recursive binding and superposition operations. The proposed transformation was experimentally investigated with symbolic strings used for modeling human perception of word similarity. The obtained results are on a par with more sophisticated approaches from the literature. The proposed transformation was designed for the HDC/VSA model known as Fourier Holographic Reduced Representations. However, it can be adapted to some other HDC/VSA models.

show abstract

Section: A Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Recursive Binding for Similarity-Preserving Hypervector Representations of Sequences

Rachkovskij¹,

Kleyko²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…This article is Part II of the survey of a research area known under the names Hyperdimensional Computing, HDC (the term was introduced in [Kanerva, 2009]) and Vector Symbolic Architectures, VSA (the term was introduced in [Gayler, 2003]). As in Part I [Kleyko et al, 2021c], below we will consistently use the joint name HDC/VSA when referring to the area. HDC/VSA is an umbrella term for the family of computational models that rely on mathematical properties of high-dimensional random spaces and use high-dimensional distributed representations called hypervectors (HVs) for structured ("symbolic") representation of data, while maintaining the advantages of traditional connectionist vector distributed representations.…”

Section: Introductionmentioning

confidence: 99%

“…Holographic Reduced Representations [Plate, 1995], [Plate, 2003] is an influential HDC/VSA model that is well-known in the machine learning domain and often used to refer to the whole family. However, for the sake of consistency, we use HDC/VSA to refer to the area.Part I of this survey [Kleyko et al, 2021c] covered foundational aspects of the area, such as historical context leading to the development of HDC/VSA, key elements of any HDC/VSA model, known HDC/VSA models, and transforming input data of various types into high-dimensional vectors suitable for HDC/VSA. This second part surveys existing applications, the role of HDC/VSA in cognitive computing and architectures, as well as directions for future work.…”

mentioning

confidence: 99%

“…Part I of this survey [Kleyko et al, 2021c] covered foundational aspects of the area, such as historical context leading to the development of HDC/VSA, key elements of any HDC/VSA model, known HDC/VSA models, and transforming input data of various types into high-dimensional vectors suitable for HDC/VSA. This second part surveys existing applications, the role of HDC/VSA in cognitive computing and architectures, as well as directions for future work.…”

mentioning

confidence: 99%

See 1 more Smart Citation

A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges

Kleyko,

Rachkovskij,

Osipov

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

This is Part II of the two-part comprehensive survey devoted to a computing framework most commonly known under the names Hyperdimensional Computing and Vector Symbolic Architectures (HDC/VSA). Both names refer to a family of computational models that use high-dimensional distributed representations and rely on the algebraic properties of their key operations to incorporate the advantages of structured symbolic representations and vector distributed representations. Holographic Reduced Representations [Plate, 1995], [Plate, 2003] is an influential HDC/VSA model that is well-known in the machine learning domain and often used to refer to the whole family. However, for the sake of consistency, we use HDC/VSA to refer to the area.Part I of this survey [Kleyko et al., 2021c] covered foundational aspects of the area, such as historical context leading to the development of HDC/VSA, key elements of any HDC/VSA model, known HDC/VSA models, and transforming input data of various types into high-dimensional vectors suitable for HDC/VSA. This second part surveys existing applications, the role of HDC/VSA in cognitive computing and architectures, as well as directions for future work. Most of the applications lie within the machine learning/artificial intelligence domain, however we also cover other applications to provide a thorough picture. The survey is written to be useful for both newcomers and practitioners.

show abstract

Integrating complex valued hyperdimensional computing with modular artificial neural networks

McDonald

Loomis

Davis

et al. 2023

Disruptive Technologies in Information Sciences VII

View full text Add to dashboard Cite

Traditional approaches using Deep Neural Networks for classification, while unquestionably successful, struggle with more general intelligence tasks such as "on the fly" learning as demonstrated by biological systems. Organisms possess myriad sensory organs for interacting with their environment. By the time these diverse sensory signals reach the brain; however, they are all converted into a spiking information representation, over which the brain itself operates. In a similar manner, myriad machine learning (ML) algorithms today compute on equally diverse data modalities; but without a consistent information representation for their respective outputs, these algorithms are frequently used independently of each other. Consequently, there is growing interest in information representations to unify these algorithms, with the larger goal of designing ML modules that may be arbitrarily arranged to solve larger-scale ML problems, analogous to digital circuit design today.One promising information representation is that of a "symbol" expressed as a high-dimensional vector, thousands of elements long. Hyperdimensional computing (HDC) is an algebra for the creation, manipulation, and measurement of correlations among "symbols" expressed as hypervectors. Towards this goal, an external plexiform layer (EPL) network, echo state network (ESN), and modern Hopfield network were adapted to implement the mathematical operations of complex phasor based HDC. Further, since symbol error correction is an important consideration for computing with networks of ML modules, a task agnostic minimum query similarity for complete symbol error correction was measured as a function of hypervector length. Based on these results, problem-independent similarities have been established within which HDC equations should be designed. Lastly, these ANNs were tested against several tasks representative of online and "plug & play" ML among expeditionary robots. For all criteria considered, the modern Hopfield network was the most capable ANN evaluated for use with complex phasor based HDC, providing 100% symbol recovery in a single time step for nearly all parameter settings.

show abstract

A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part I: Models and Data Transformations

Cited by 5 publications

References 43 publications

Recursive Binding for Similarity-Preserving Hypervector Representations of Sequences

Recursive Binding for Similarity-Preserving Hypervector Representations of Sequences

A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges

Integrating complex valued hyperdimensional computing with modular artificial neural networks

Contact Info

Product

Resources

About