Modeling spatial–temporal operations with context-dependent associative memories

Mizraji, Eduardo; Lin, Juan

doi:10.1007/s11571-015-9343-3

Cited by 8 publications

(5 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…encode complicated topological relationships that the brain is capable of computing. An approach to this difficult problem has been published by Mizraji and Lin ( 2015 ) based on the computational capabilities of multiplicative contexts. In that article, the authors present a hierarchical model with three neural layers, ranging from concrete natural language phrases to increasingly abstract and general encodings.…”

Section: Multiplicative Contexts In Matrix Memoriesmentioning

confidence: 99%

Multiplicative processing in the modeling of cognitive activities in large neural networks

Valle-Lisboa

Mizraji

2023

Biophys Rev

Self Cite

View full text Add to dashboard Cite

Explaining the foundation of cognitive abilities in the processing of information by neural systems has been in the beginnings of biophysics since McCulloch and Pitts pioneered work within the biophysics school of Chicago in the 1940s and the interdisciplinary cybernetists meetings in the 1950s, inseparable from the birth of computing and artificial intelligence. Since then, neural network models have traveled a long path, both in the biophysical and the computational disciplines. The biological, neurocomputational aspect reached its representational maturity with the Distributed Associative Memory models developed in the early 70 s. In this framework, the inclusion of signal-signal multiplication within neural network models was presented as a necessity to provide matrix associative memories with adaptive, context-sensitive associations, while greatly enhancing their computational capabilities. In this review, we show that several of the most successful neural network models use a form of multiplication of signals. We present several classical models that included such kind of multiplication and the computational reasons for the inclusion. We then turn to the different proposals about the possible biophysical implementation that underlies these computational capacities. We pinpoint the important ideas put forth by different theoretical models using a tensor product representation and show that these models endow memories with the context-dependent adaptive capabilities necessary to allow for evolutionary adaptation to changing and unpredictable environments. Finally, we show how the powerful abilities of contemporary computationally deep-learning models, inspired in neural networks, also depend on multiplications, and discuss some perspectives in view of the wide panorama unfolded. The computational relevance of multiplications calls for the development of new avenues of research that uncover the mechanisms our nervous system uses to achieve multiplication.

show abstract

Section: Multiplicative Contexts In Matrix Memoriesmentioning

confidence: 99%

Multiplicative processing in the modeling of cognitive activities in large neural networks

Valle-Lisboa

Mizraji

2023

Biophys Rev

Self Cite

View full text Add to dashboard Cite

show abstract

“…Among the others, the semantic level can interact with the acoustic level (Li et al 2017) and with the syntactic one (Malaia and Newman 2015) and both aspects are important to allocate attentional resources. Mizraji and Lin (2015) stressed the importance of coding spatial and temporal relations in autoassociative networks, and emphasized the role of the context. Finally, some authors stressed the need for a symbolic level, between the physiological and the cognitive ones (Bonzon 2017).…”

Section: Limitations Of the Modelmentioning

confidence: 99%

A feature-based neurocomputational model of semantic memory

et al. 2018

View full text Add to dashboard Cite

According with a featural organization of semantic memory, this work is aimed at investigating, through an attractor network, the role of different kinds of features in the representation of concepts, both in normal and neurodegenerative conditions. We implemented new synaptic learning rules in order to take into account the role of partially shared features and of distinctive features with different saliency. The model includes semantic and lexical layers, coding, respectively for object features and word-forms. Connections among nodes are strongly asymmetrical. To account for the feature saliency, asymmetrical synapses were created using Hebbian rules of potentiation and depotentiation, setting different pre-synaptic and post-synaptic thresholds. A variable post-synaptic threshold, which automatically changed to reflect the feature frequency in different concepts (i.e., how many concepts share a feature), was used to account for partially shared features. The trained network solved naming tasks and word recognition tasks very well, exploiting the different role of salient versus marginal features in concept identification. In the case of damage, superordinate concepts were preserved better than the subordinate ones. Interestingly, the degradation of salient features, but not of marginal ones, prevented object identification. The model suggests that Hebbian rules, with adjustable post-synaptic thresholds, can provide a reliable semantic representation of objects exploiting the statistics of input features.

show abstract

“…This is an important assessment of learning of higher-order representations of an input-output relation given a specific context, not reducible to pair associations. Statistical learning of such context-dependent representations is addressed by models based on tensor products of vectors by associating an output to the Kronecker product of an input and a context (e.g., Mizraji and Lin 2015;Mizraji et al 2009). An important issue for cognitive neuroscience is to account for context-dependent processing in terms of biologically realistic mechanisms at the neural level.…”

Section: Multiple Priming and The Limits Of Hebbian Learningmentioning

confidence: 99%

Semantic integration by pattern priming: experiment and cortical network model

et al. 2016

View full text Add to dashboard Cite

Neural network models describe semantic priming effects by way of mechanisms of activation of neurons coding for words that rely strongly on synaptic efficacies between pairs of neurons. Biologically inspired Hebbian learning defines efficacy values as a function of the activity of pre-and post-synaptic neurons only. It generates only pair associations between words in the semantic network. However, the statistical analysis of large text databases points to the frequent occurrence not only of pairs of words (e.g., ''the way'') but also of patterns of more than two words (e.g., ''by the way''). The learning of these frequent patterns of words is not reducible to associations between pairs of words but must take into account the higher level of coding of three-word patterns. The processing and learning of pattern of words challenges classical Hebbian learning algorithms used in biologically inspired models of priming. The aim of the present study was to test the effects of patterns on the semantic processing of words and to investigate how an inter-synaptic learning algorithm succeeds at reproducing the experimental data. The experiment manipulates the frequency of occurrence of patterns of three words in a multiple-paradigm protocol. Results show for the first time that target words benefit more priming when embedded in a pattern with the two primes than when only associated with each prime in pairs. A biologically inspired inter-synaptic learning algorithm is tested that potentiates synapses as a function of the activation of more than two pre-and postsynaptic neurons. Simulations show that the network can learn patterns of three words to reproduce the experimental results.

show abstract

Modeling spatial–temporal operations with context-dependent associative memories

Cited by 8 publications

References 52 publications

Multiplicative processing in the modeling of cognitive activities in large neural networks

Multiplicative processing in the modeling of cognitive activities in large neural networks

A feature-based neurocomputational model of semantic memory

Semantic integration by pattern priming: experiment and cortical network model

Contact Info

Product

Resources

About