Liquid computing on and off the edge of chaos with a striatal microcircuit

Toledo-Suárez, Carlos D.; Duarte, Renato; Morrison, Abigail

doi:10.3389/fncom.2014.00130

Cited by 6 publications

(2 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These percentage connectivity values worked the best in terms of accuracy, for the neuron parameter selections in this work shown in Table 2. The strengths of all the connections (W∈[0,1]NPRE×NPOST) were selected randomly (Maass et al, 2002) from a uniform distribution U (0, 1) (Toledo-Suárez et al, 2014; Srinivasan et al, 2018). A randomly generated mask (M∈{0,1}NPRE×NPOST,mij∈M) decides which connections exist to obtain the desired sparsity/percentage connectivity (PPRE→POST=msub∑(∀i,∀j)mij(NPRE×NPOST)×100%).…”

Section: Methodsmentioning

confidence: 99%

Analysis of Liquid Ensembles for Enhancing the Performance and Accuracy of Liquid State Machines

et al. 2019

View full text Add to dashboard Cite

Liquid state machine (LSM), a bio-inspired computing model consisting of the input sparsely connected to a randomly interlinked reservoir (or liquid) of spiking neurons followed by a readout layer, finds utility in a range of applications varying from robot control and sequence generation to action, speech, and image recognition. LSMs stand out among other Recurrent Neural Network (RNN) architectures due to their simplistic structure and lower training complexity. Plethora of recent efforts have been focused toward mimicking certain characteristics of biological systems to enhance the performance of modern artificial neural networks. It has been shown that biological neurons are more likely to be connected to other neurons in the close proximity, and tend to be disconnected as the neurons are spatially far apart. Inspired by this, we propose a group of locally connected neuron reservoirs, or an ensemble of liquids approach, for LSMs. We analyze how the segmentation of a single large liquid to create an ensemble of multiple smaller liquids affects the latency and accuracy of an LSM. In our analysis, we quantify the ability of the proposed ensemble approach to provide an improved representation of the input using the Separation Property (SP) and Approximation Property (AP). Our results illustrate that the ensemble approach enhances class discrimination (quantified as the ratio between the SP and AP), leading to better accuracy in speech and image recognition tasks, when compared to a single large liquid. Furthermore, we obtain performance benefits in terms of improved inference time and reduced memory requirements, due to lowered number of connections and the freedom to parallelize the liquid evaluation process.

show abstract

Section: Methodsmentioning

confidence: 99%

Analysis of Liquid Ensembles for Enhancing the Performance and Accuracy of Liquid State Machines

et al. 2019

View full text Add to dashboard Cite

show abstract

“…However, it is not obvious how the environmental states are represented in the reinforcement learning of the basal ganglia. Based on the hypothesis that the striatum responds to diverse inputs from different cortical sources and plays a computational role for discriminating inputs in reinforcement learning-based decision making, the LSM properties of a striatal microcircuit were studied with computational models (Toledo-Suárez et al, 2014). It was demonstrated that the separation and approximation properties required for the LSM are generated using a model network of medium spiny neurons and fast spiking interneurons coupled via inhibitory synapses in a supervised learning task.…”

Section: Brain Regionsmentioning

confidence: 99%

Recent advances in physical reservoir computing: A review

et al. 2019

View full text Add to dashboard Cite

Reservoir computing is a computational framework suited for temporal/sequential data processing. It is derived from several recurrent neural network models, including echo state networks and liquid state machines. A reservoir computing system consists of a reservoir for mapping inputs into a high-dimensional space and a readout for pattern analysis from the high-dimensional states in the reservoir. The reservoir is fixed and only the readout is trained with a simple method such as linear regression and classification. Thus, the major advantage of reservoir computing compared to other recurrent neural networks is fast learning, resulting in low training cost. Another advantage is that the reservoir without adaptive updating is amenable to hardware implementation using a variety of physical systems, substrates, and devices. In fact, such physical reservoir computing has attracted increasing attention in diverse fields of research. The purpose of this review is to provide an overview of recent advances in physical reservoir computing by classifying them according to the type of the reservoir. We discuss the current issues and perspectives related to physical reservoir computing, in order to further expand its practical applications and develop next-generation machine learning systems.

show abstract

Driving reservoir models with oscillations: a solution to the extreme structural sensitivity of chaotic networks

2016

View full text Add to dashboard Cite

A large body of experimental and theoretical work on neural coding suggests that the information stored in brain circuits is represented by time-varying patterns of neural activity. Reservoir computing, where the activity of a recurrently connected pool of neurons is read by one or more units that provide an output response, successfully exploits this type of neural activity. However, the question of system robustness to small structural perturbations, such as failing neurons and synapses, has been largely overlooked. This contrasts with well-studied dynamical perturbations that lead to divergent network activity in the presence of chaos, as is the case for many reservoir networks. Here, we distinguish between two types of structural network perturbations, namely local (e.g., individual synaptic or neuronal failure) and global (e.g., network-wide fluctuations). Surprisingly, we show that while global perturbations have a limited impact on the ability of reservoir models to perform various tasks, local perturbations can produce drastic effects. To address this limitation, we introduce a new architecture where the reservoir is driven by a layer of oscillators that generate stable and repeatable trajectories. This model outperforms previous implementations while being resistant to relatively large local and global perturbations. This finding has implications for the design of reservoir models that capture the capacity of brain circuits to perform cognitively and behaviorally relevant tasks while remaining robust to various forms of perturbations. Further, our work proposes a novel role for neuronal oscillations found in cortical circuits, where they may serve as a collection of inputs from which a network can robustly generate complex dynamics and implement rich computations.

show abstract

Liquid computing on and off the edge of chaos with a striatal microcircuit

Cited by 6 publications

References 53 publications

Analysis of Liquid Ensembles for Enhancing the Performance and Accuracy of Liquid State Machines

Analysis of Liquid Ensembles for Enhancing the Performance and Accuracy of Liquid State Machines

Recent advances in physical reservoir computing: A review

Driving reservoir models with oscillations: a solution to the extreme structural sensitivity of chaotic networks

Contact Info

Product

Resources

About