A Computational Theory for Life-Long Learning of Semantics

Sutor, Peter; Summers-Stay, Douglas; Aloimonos, Yiannis

doi:10.1007/978-3-319-97676-1_21

Cited by 8 publications

(13 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This may become expensive when the hyperdimensional space contain many concepts. In order to maintain that data of a particular modality is closer to other examples of that modality, it may be necessary to adopt an approach that facilitates this, such as in Sutor et al ( 2018 ).…”

Section: Discussionmentioning

confidence: 99%

Symbolic Representation and Learning With Hyperdimensional Computing

et al. 2020

Self Cite

View full text Add to dashboard Cite

It has been proposed that machine learning techniques can benefit from symbolic representations and reasoning systems. We describe a method in which the two can be combined in a natural and direct way by use of hyperdimensional vectors and hyperdimensional computing. By using hashing neural networks to produce binary vector representations of images, we show how hyperdimensional vectors can be constructed such that vector-symbolic inference arises naturally out of their output. We design the Hyperdimensional Inference Layer (HIL) to facilitate this process and evaluate its performance compared to baseline hashing networks. In addition to this, we show that separate network outputs can directly be fused at the vector symbolic level within HILs to improve performance and robustness of the overall model. Furthermore, to the best of our knowledge, this is the first instance in which meaningful hyperdimensional representations of images are created on real data, while still maintaining hyperdimensionality.

show abstract

Section: Discussionmentioning

confidence: 99%

Symbolic Representation and Learning With Hyperdimensional Computing

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…This work is an extension of a series of works. First, [4]- [8] describe some methods of encoding arbitrary data into functional hypervectors. Namely, ideas presented in [7] and [8] were used to facilitate the process of converting output signals from networks into hypervectors.…”

Section: A Prior Workmentioning

confidence: 99%

Gluing Neural Networks Symbolically Through Hyperdimensional Computing

Sutor¹,

Yuan²,

Summers-Stay³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Hyperdimensional Computing affords simple, yet powerful operations to create long Hyperdimensional Vectors (hypervectors) that can efficiently encode information, be used for learning, and are dynamic enough to be modified on the fly. In this paper, we explore the notion of using binary hypervectors to directly encode the final, classifying output signals of neural networks in order to fuse differing networks together at the symbolic level. This allows multiple neural networks to work together to solve a problem, with little additional overhead. Output signals just before classification are encoded as hypervectors and bundled together through consensus summation to train a classification hypervector. This process can be performed iteratively and even on single neural networks by instead making a consensus of multiple classification hypervectors. We find that this outperforms the state of the art, or is on a par with it, while using very little overhead, as hypervector operations are extremely fast and efficient in comparison to the neural networks. This consensus process can learn online and even grow or lose models in real time. Hypervectors act as memories that can be stored, and even further bundled together over time, affording life long learning capabilities. Additionally, this consensus structure inherits the benefits of Hyperdimensional Computing, without sacrificing the performance of modern Machine Learning. This technique can be extrapolated to virtually any neural model, and requires little modification to employ -one simply requires recording the output signals of networks when presented with a testing example.

show abstract

“…On the other hand, if there is data associated with a problem an alternative way to mappings would be to obtain atomic HD vectors with the help of the available data via, e.g., an optimization process. For example, in [1] the optimizationbased mapping [58] was used. Fig.…”

Section: Data Representation In Vsas a Atomic Representationsmentioning

confidence: 99%

“…It rather shows that similar distance structures can be obtained in different ways. Last, it is worth mentioning that the optimization-based method [58] for obtaining similarity preserving HD vectors can be contrasted to a Random Indexing method [35]. Random Indexing also implicitly (i.e., without constructing the co-occurrence matrix) uses co-occurrence statistics in order to form similarity preserving HD vectors based on available data.…”

Section: Data Representation In Vsas a Atomic Representationsmentioning

confidence: 99%

“…Random Indexing also implicitly (i.e., without constructing the co-occurrence matrix) uses co-occurrence statistics in order to form similarity preserving HD vectors based on available data. The difference, however, is that Random Indexing is optimization-free as it forms representations through a single pass via the data, and, thus, it can be done on-line while the optimization required by [58] calls for off-line processing. It is an interesting research question whether these methods arrive at similar representations in terms of inter-item distance structure or whether the optimization process brings extra benefits by refining the inter-item distance structure.…”

Section: Data Representation In Vsas a Atomic Representationsmentioning

confidence: 99%

See 1 more Smart Citation

Commentaries on "Learning Sensorimotor Control with Neuromorphic Sensors: Toward Hyperdimensional Active Perception" [Science Robotics Vol. 4 Issue 30 (2019) 1-10

Kleyko,

Gayler,

Osipov

2020

Preprint

View full text Add to dashboard Cite

This correspondence comments on the findings reported in a recent Science Robotics article by Mitrokhin et al. [1]. The main goal of this commentary is to expand on some of the issues touched on in that article. Our experience is that hyperdimensional computing is very different from other approaches to computation and that it can take considerable exposure to its concepts before attaining practically useful understanding. Therefore, in order to provide an overview of the area to the first time reader of [1], the commentary includes a brief historic overview as well as connects the findings of the article to a larger body of literature existing in the area.

show abstract

A Computational Theory for Life-Long Learning of Semantics

Cited by 8 publications

References 9 publications

Symbolic Representation and Learning With Hyperdimensional Computing

Symbolic Representation and Learning With Hyperdimensional Computing

Gluing Neural Networks Symbolically Through Hyperdimensional Computing

Commentaries on "Learning Sensorimotor Control with Neuromorphic Sensors: Toward Hyperdimensional Active Perception" [Science Robotics Vol. 4 Issue 30 (2019) 1-10

Contact Info

Product

Resources

About