Representing sets as summed semantic vectors

Summers-Stay, Douglas; Sutor, Peter; Li, Dandan

doi:10.1016/j.bica.2018.07.002

Cited by 6 publications

(6 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus, there is a need for a general technique to project real-valued embeddings from data-driven systems to binary spaces. As a result, real-value hyperdimensional vectors may be better suited to certain tasks (Summers-Stay et al, 2018 ; Sutor et al, 2019 ).…”

Section: Discussionmentioning

confidence: 99%

“…On the solely symbolic representation and reasoning side, there exists relevant work on using cellular automata based hyperdimensional computing (Yilmaz, 2015 ). Some formulations based on real-valued vectors can also exhibit similar properties to long binary vectors so far as compositionality and decompositionality is concerned (Summers-Stay et al, 2018 ).…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Symbolic Representation and Learning With Hyperdimensional Computing

et al. 2020

Self Cite

View full text Add to dashboard Cite

It has been proposed that machine learning techniques can benefit from symbolic representations and reasoning systems. We describe a method in which the two can be combined in a natural and direct way by use of hyperdimensional vectors and hyperdimensional computing. By using hashing neural networks to produce binary vector representations of images, we show how hyperdimensional vectors can be constructed such that vector-symbolic inference arises naturally out of their output. We design the Hyperdimensional Inference Layer (HIL) to facilitate this process and evaluate its performance compared to baseline hashing networks. In addition to this, we show that separate network outputs can directly be fused at the vector symbolic level within HILs to improve performance and robustness of the overall model. Furthermore, to the best of our knowledge, this is the first instance in which meaningful hyperdimensional representations of images are created on real data, while still maintaining hyperdimensionality.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Symbolic Representation and Learning With Hyperdimensional Computing

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…In our previous work we have focused on the use of binary hypervectors to perform the binding and unbinding, based on the fact that conventional binding of real valued vectors using binding techniques such as Holographic Reduced Representation (HRR) 7,8,24 are limited in the number of hypervectors that can be bound and unbound in a way that scales with the square root of the vector dimension. In 25,26 an alternative approach to binding and unbinding real valued vectors, that potentially offers greater binding capacity have been demonstrated and evaluation of this approach is part of our current research. In this paper we therefore compare the capability to map both bundled real valued vectors and bundled binary hypervectors.…”

Section: Vector Binding and Unbindingmentioning

confidence: 99%

Semantic vector space mapping in edge of network multi-domain operations

Bent¹,

Summers-Stay

Preece

et al. 2023

Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications V

View full text Add to dashboard Cite

Artificial Intelligence applications are increasingly making use of vector embedding techniques to achieve impressive results in many application domains. Semantic Vector Spaces (SVS's) are constructed using semantic vector embedding techniques that learn vector representations of data across multiple domains. An important application area enabled by such techniques is the capability to represent software services and service workflows as semantic hypervectors. Previous work has shown how these hypervector representations have significant advantages over alternative schemes for decentralized service workflow construction, particularly in low communications bandwidth and energy constrained environments that are typical in multi-domain operations. SVS construction usually assumes that all the data required to construct the semantic vector space is available centrally. However, in multi-domain operations different partners may not be willing to share the training data necessary to construct a common multi-domain SVS. Hence semantic hypervectors representing similar services or workflows but constructed from different training data by different partners cannot be discovered and used. In this paper we focus on how it is possible to map semantic hypervectors between partner SVS's, so that complementary services and workflows developed and owned by different partners can be discovered and used to achieve mission goals. The paper describes techniques for generating the required mapping that require a minimum exchange of information between the different partners; demonstrate how it is possible to do this for semantic hypervectors that use different types of encoding (eg., real valued, binary, sparse slot-encoding); and illustrates how the mapping can be implemented in various multi-domain operational settings.

show abstract

“…Some of these results can be obtained analytically using the capacity theory. Additionally, [Summers-Stay et al, 2018], [Frady et al, 2018b], [Kim, 2018], [Hersche et al, 2021] elaborated on methods for recovering information from compositional HVs beyond the standard nearest neighbor search in the item memory, reaching to the capacity of up to 1.2 bits/component [Hersche et al, 2021]. The works above were focused on the case where a single HV was used to store information but as it was demonstrated in [Danihelka et al, 2016] the decoding from HVs can be improved if the redundant storage is used.…”

Section: Information Capacity Of Hvsmentioning

confidence: 99%

A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part I: Models and Data Transformations

Kleyko,

Rachkovskij,

Osipov

et al. 2021

Preprint

View full text Add to dashboard Cite

This two-part comprehensive survey is devoted to a computing framework most commonly known under the names Hyperdimensional Computing and Vector Symbolic Architectures (HDC/VSA). Both names refer to a family of computational models that use highdimensional distributed representations and rely on the algebraic properties of their key operations to incorporate the advantages of structured symbolic representations and vector distributed representations. Notable models in the HDC/VSA family are Tensor Product Representations, Holographic Reduced Representations, Multiply-Add-Permute, Binary Spatter Codes, and Sparse Binary Distributed Representations but there are other models too. HDC/VSA is a highly interdisciplinary area with connections to computer science, electrical engineering, artificial intelligence, mathematics, and cognitive science. This fact makes it challenging to create a thorough overview of the area. However, due to a surge of new researchers joining the area in recent years, the necessity for a comprehensive survey of the area has become extremely important. Therefore, amongst other aspects of the area, this Part I surveys important aspects such as: known computational models of HDC/VSA and transformations of various input data types to high-dimensional distributed representations. Part II of this survey [Kleyko et al., 2021c] is devoted to applications, cognitive computing and architectures, as well as directions for future work. The survey is written to be useful for both newcomers and practitioners.

show abstract

Representing sets as summed semantic vectors

Cited by 6 publications

References 14 publications

Symbolic Representation and Learning With Hyperdimensional Computing

Symbolic Representation and Learning With Hyperdimensional Computing

Semantic vector space mapping in edge of network multi-domain operations

A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part I: Models and Data Transformations

Contact Info

Product

Resources

About