SpaRCe: Improved Learning of Reservoir Computing Systems Through Sparse Representations

Manneschi, Luca; Lin, Andrew C.; Vasilaki, Eleni

doi:10.1109/tnnls.2021.3102378

Cited by 21 publications

(8 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, activity-dependent compensation may provide useful techniques for machine learning. For example, we found that performance of a reservoir computing network could be improved if thresholds of individual neurons are initialized to achieve a particular activity probability given the distribution of input activities ( 65 ).…”

Section: Discussionmentioning

confidence: 99%

Compensatory variability in network parameters enhances memory performance in the Drosophila mushroom body

Abdelrahman

Vasilaki

Lin

2021

Proc. Natl. Acad. Sci. U.S.A.

Self Cite

View full text Add to dashboard Cite

Neural circuits use homeostatic compensation to achieve consistent behavior despite variability in underlying intrinsic and network parameters. However, it remains unclear how compensation regulates variability across a population of the same type of neurons within an individual and what computational benefits might result from such compensation. We address these questions in the Drosophila mushroom body, the fly’s olfactory memory center. In a computational model, we show that under sparse coding conditions, memory performance is degraded when the mushroom body’s principal neurons, Kenyon cells (KCs), vary realistically in key parameters governing their excitability. However, memory performance is rescued while maintaining realistic variability if parameters compensate for each other to equalize KC average activity. Such compensation can be achieved through both activity-dependent and activity-independent mechanisms. Finally, we show that correlations predicted by our model’s compensatory mechanisms appear in the Drosophila hemibrain connectome. These findings reveal compensatory variability in the mushroom body and describe its computational benefits for associative memory.

show abstract

Section: Discussionmentioning

confidence: 99%

Compensatory variability in network parameters enhances memory performance in the Drosophila mushroom body

Abdelrahman

Vasilaki

Lin

2021

Proc. Natl. Acad. Sci. U.S.A.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Adding recurrent connections to a layer in a network can enable reverberation of activity, which can be a form of memory in the network [Manneschi et al, 2019[Manneschi et al, , 2021. Recurrent pathways can transform the kinds of classification problems the network is able to solve [Cope et al, 2018].…”

Section: Patterns Of Connectivity: Reverberation and Processing Acros...mentioning

confidence: 99%

The Relationship between Cognition and Brain Size or Neuron Number

Barron¹,

Mourmourakis²

2023

Brain Behav Evol

View full text Add to dashboard Cite

The comparative approach is a powerful way to explore the relationship between brain structure and cognitive function. Thus far the field has been dominated by the assumption that a bigger brain somehow means better cognition. Correlations between differences in brain size or neuron number between species and differences in specific cognitive abilities exist, but these correlations are very noisy. Extreme differences exist between clades in the relationship between either brain size or neuron number and specific cognitive abilities. This means that correlations become weaker, not stronger, as the taxonomic diversity of sampled groups increases. Cognition is the outcome of neural networks. Here we propose that considering plausible neural network models will advance our understanding of the complex relationships between neuron number and different aspects of cognition. Computational modelling of networks suggests that adding pathways, or layers, or changing patterns of connectivity in a network can all have different specific consequences for cognition. Consequently, models of computational architecture can help us hypothesise how and why differences in neuron number might be related to differences in cognition. As methods in connectomics continue to improve and more structural information on animal brains becomes available we are learning more about natural network structures in brains, and we can develop more biologically plausible models of cognitive architecture. Natural animal diversity then becomes a powerful resource to both test the assumptions of these models and explore hypotheses for how neural network structure and network size might delimit cognitive function.

show abstract

“…Here we employ short training-datasets (225 data points) to mimic real-world applications with strict data-collection time and energy requirements. To avoid overfitting, we employ a sophisticated feature-selection algorithm (each FMR output channel is considered a single computational 'feature' 62,76 ) with 10-fold cross validation (see Methods). Performance is evaluated via the mean squared error (MSE) between the reservoir-prediction and target.…”

Section: /24mentioning

confidence: 99%

Adaptive Programmable Networks for In Materia Neuromorphic Computing

Stenning

Gartside

Manneschi

et al. 2022

Preprint

View full text Add to dashboard Cite

Nanomagnetic artificial spin-systems are ideal candidates for neuromorphic hardware. Their passive memory, state-dependent dynamics and nonlinear GHz spin-wave response provide powerful computation. However, any single physical reservoir must trade-off between performance metrics including nonlinearity and memory-capacity, with the compromise typically hard-coded. Here, we present three artificial spin-systems and show how tuning system geometry and dynamics defines computing performance. We engineer networks where each node is a high-dimensional physical reservoir, implementing parallel, deep and multilayer physical neural network architectures. This solves the issue of physical reservoir performance compromise, allowing a small suite of synergistic physical systems to address diverse tasks and provide a broad range of reprogrammable computationally-distinct configurations. These networks outperform any single reservoir across a broad taskset. Crucially, we move beyond reservoir computing to present a method for reconfigurably programming inter-layer network connections, enabling on-demand task optimised performance.

show abstract

SpaRCe: Improved Learning of Reservoir Computing Systems Through Sparse Representations

Cited by 21 publications

References 59 publications

Compensatory variability in network parameters enhances memory performance in the Drosophila mushroom body

Compensatory variability in network parameters enhances memory performance in the Drosophila mushroom body

The Relationship between Cognition and Brain Size or Neuron Number

Adaptive Programmable Networks for In Materia Neuromorphic Computing

Contact Info

Product

Resources

About