Vector Symbolic Architectures for Context-Free Grammars

Graben, Peter beim; Huber, Markus; Meyer, Werner; Römer, Ronald; Wolff, Matthias

doi:10.1007/s12559-021-09974-y

Cited by 5 publications

(2 citation statements)

References 67 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Transformations of pushdown automata and context-free grammars into HVs have been presented in [12]. A proposal for implementing Turing machines and cellular automata was given in [204].…”

Section: Automata Instructions and Schemasmentioning

confidence: 99%

See 1 more Smart Citation

A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges

et al. 2023

View full text Add to dashboard Cite

This is Part II of the two-part comprehensive survey devoted to a computing framework most commonly known under the names Hyperdimensional Computing and Vector Symbolic Architectures (HDC/VSA). Both names refer to a family of computational models that use high-dimensional distributed representations and rely on the algebraic properties of their key operations to incorporate the advantages of structured symbolic representations and vector distributed representations. Holographic Reduced Representations [322, 327] is an influential HDC/VSA model that is well-known in the machine learning domain and often used to refer to the whole family. However, for the sake of consistency, we use HDC/VSA to refer to the field. Part I of this survey [223] covered foundational aspects of the field, such as the historical context leading to the development of HDC/VSA, key elements of any HDC/VSA model, known HDC/VSA models, and the transformation of input data of various types into high-dimensional vectors suitable for HDC/VSA. This second part surveys existing applications, the role of HDC/VSA in cognitive computing and architectures, as well as directions for future work. Most of the applications lie within the Machine Learning/Artificial Intelligence domain, however, we also cover other applications to provide a complete picture. The survey is written to be useful for both newcomers and practitioners.

show abstract

“…Transformations of pushdown automata and context-free grammars into HVs have been presented in [12]. A proposal for implementing Turing machines and cellular automata was given in [204].…”

Section: Automata Instructions and Schemasmentioning

confidence: 99%

“…A related research direction was initiated by beim Graben and colleagues [11][12][13]421]. It concerns establishing a mathematically rigorous formalism of Tensor Product Representations using the concept of a Fock space [83].…”

Section: Automata Instructions and Schemasmentioning

confidence: 99%

A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges

et al. 2023

View full text Add to dashboard Cite

show abstract

Multiplicative processing in the modeling of cognitive activities in large neural networks

Valle-Lisboa

Mizraji

2023

Biophys Rev

View full text Add to dashboard Cite

Explaining the foundation of cognitive abilities in the processing of information by neural systems has been in the beginnings of biophysics since McCulloch and Pitts pioneered work within the biophysics school of Chicago in the 1940s and the interdisciplinary cybernetists meetings in the 1950s, inseparable from the birth of computing and artificial intelligence. Since then, neural network models have traveled a long path, both in the biophysical and the computational disciplines. The biological, neurocomputational aspect reached its representational maturity with the Distributed Associative Memory models developed in the early 70 s. In this framework, the inclusion of signal-signal multiplication within neural network models was presented as a necessity to provide matrix associative memories with adaptive, context-sensitive associations, while greatly enhancing their computational capabilities. In this review, we show that several of the most successful neural network models use a form of multiplication of signals. We present several classical models that included such kind of multiplication and the computational reasons for the inclusion. We then turn to the different proposals about the possible biophysical implementation that underlies these computational capacities. We pinpoint the important ideas put forth by different theoretical models using a tensor product representation and show that these models endow memories with the context-dependent adaptive capabilities necessary to allow for evolutionary adaptation to changing and unpredictable environments. Finally, we show how the powerful abilities of contemporary computationally deep-learning models, inspired in neural networks, also depend on multiplications, and discuss some perspectives in view of the wide panorama unfolded. The computational relevance of multiplications calls for the development of new avenues of research that uncover the mechanisms our nervous system uses to achieve multiplication.

show abstract

Opening the Black Box: Finding Osgood’s Semantic Factors in Word2vec Space

Surov

2022

View full text Add to dashboard Cite

State-of-the-art models of artificial intelligence are developed in the black-box paradigm, in which sensitive information is limited to input-output interfaces, while internal representations are not interpretable. The resulting algorithms lack explainability and transparency, requested for responsible application. This paper addresses the problem by a method for finding Osgood’s dimensions of affective meaning in multidimensional space of a pre-trained word2vec model of natural language. Three affective dimensions are found based on eight semantic prototypes, composed of individual words. Evaluation axis is found in 300-dimensional word2vec space as a difference between positive and negative prototypes. Potency and activity axes are defined from six process-semantic prototypes (perception, analysis, planning, action, progress, and evaluation), representing phases of a generalized circular process in that plane. All dimensions are found in simple analytical form, not requiring additional training. Dimensions are nearly orthogonal, as expected for independent semantic factors. Osgood’s semantics of any word2vec object is then retrieved by a simple projection of the corresponding vector to the identified dimensions. The developed approach opens the possibility for interpreting the inside of black box-type algorithms in natural affective-semantic categories, and provides insights into foundational principles of distributive vector models of natural language. In the reverse direction, the established mapping opens machine-learning models as rich sources of data for cognitive-behavioral research and technology.

show abstract

Vector Symbolic Architectures for Context-Free Grammars

Cited by 5 publications

References 67 publications

A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges

A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges

Multiplicative processing in the modeling of cognitive activities in large neural networks

Opening the Black Box: Finding Osgood’s Semantic Factors in Word2vec Space

Contact Info

Product

Resources

About