How neural circuits achieve and use stable dynamics

Abstract

11The brain consists of many interconnected networks with time-varying activity. There 12 are multiple sources of noise and variation yet activity has to eventually converge to a 13 stable state for its computations to make sense. We approached this from a control-14 theory perspective by applying contraction analysis to recurrent neural networks. This 15 allowed us to find mechanisms for achieving stability in multiple connected networks 16 with biologically realistic dynamics, including synaptic plasticity and time-varying inputs. 17These mechanisms included anti-Hebbian plasticity, synaptic sparsity and excitatory-18 inhibitory balance. We leveraged these findings to construct networks that could 19 perform functionally relevant computations in the presence of noise and disturbance. 20Our work provides a blueprint for how to construct stable plastic and distributed 21 networks. 22 23 2 2 Introduction 24 25The brain is comprised of networks that are highly dynamic and noisy. Neural activity 26 fluctuates from moment to moment and varies considerably between experimentally 27 identical trials (Latimer et al., 2015;Lundqvist et al., 2016; Churchland et al., 28 2011). These fluctuations can be due to a variety of factors including variability in 29 membrane potentials, inputs, plastic changes due to recent experience and so on. Yet, 30 in spite of these fluctuations, networks must 31 achieve computational stability. Despite 32 being "knocked around" by different starting 33 conditions and noise, networks must reach a 34 highly consistent state for their computations 35 to make sense. 36 The mechanisms that produce neural 37 network stability have been characterized 38 primarily in recurrent neural networks 39 (RNNs)--a general form of brain network-in 40 cases where the network weights are fixed 41 and the input the network receives is 42 constant (Fang and Kincaid 1996; Dayan and Abbot 2005). These stability conditions 43 are bounds on the eigenvalues of the weight matrix and prevent networks from "blowing 44 up", that is, from running away to high levels of excitation (Fang and Kincaid 1996; 45 Matsuoka 1992). This is an important finding but it is not the whole story. Eigenvalue 46 analysis of the weight matrix is only guaranteed to work in RNNs receiving constant 47 input and with fixed synaptic weights (or weights that change very slowly). Biological 48 networks, however, have plastic synaptic weights that change rapidly under constant 49 bombardment from environmental inputs. 50 Such "dynamic stability" can be studied using contraction analysis, a concept developed 51 in control theory. Unlike a chaotic system where perturbations and distortions can be 52 amplified over time, the population activity of a contracting network will converge 53 towards the same trajectory, thus achieving stable dynamics (Figure 1). One way to 54 understand contraction is to represent the state of a network at a given time as a point 55 in the network's 'state-space'. A commonly used state-space in neuroscience is the 56 space...