Developmental Emergence of Sparse Coding: A Dynamic Systems Approach

Abstract

During neocortical development, network activity undergoes a dramatic transition from largely synchronized, so-called cluster activity, to a relatively sparse pattern around the time of eye-opening in rodents. Biophysical mechanisms underlying this sparsification phenomenon remain poorly understood. Here, we present a dynamic systems modeling study of a developing neural network that provides the first mechanistic insights into sparsification. We find that the rest state of immature networks is strongly affected by the dynamics of a transient, unstable state hidden in their firing activities, allowing these networks to either be silent or generate large cluster activity. We address how, and which, specific developmental changes in neuronal and synaptic parameters drive sparsification. We also reveal how these changes refine the information processing capabilities of an in vivo developing network, mainly by showing a developmental reduction in the instability of network’s firing activity, an effective availability of inhibition-stabilized states, and an emergence of spontaneous attractors and state transition mechanisms. Furthermore, we demonstrate the key role of GABAergic transmission and depressing glutamatergic synapses in governing the spatiotemporal evolution of cluster activity. These results, by providing a strong link between experimental observations and model behavior, suggest how adult sparse coding networks may emerge developmentally.

Somatostatin Interneurons Promote Neuronal Synchrony in the Neonatal Hippocampus

“…This unstable FP, which disappears after the peak of the simGDPs, enables the emergence of simGDPs by building an amplification threshold (magenta line, Figure 5C) in the initial activity phase of the STP-RNN operating at its rest state. The simGDPs can be effectively triggered only if the perturbation is sufficiently strong enough to push activity of the STP-RNN beyond the amplification threshold ( Figure 5C) (Rahmati et al, 2017). This underlies a relatively all-or-none characteristic of simGDPs, which is a wellknown property of experimentally measured GDPs.…”

“…To investigate the mechanism of simGDP emergence, we plotted the A s À A p -plane of the network with slow STP dynamics frozen at the rest state (Frozen STP-RNN; Figure 5C; STAR Methods). We found that an unstable fixed point (FP; black dots in A s ; Figures 5C and 5E) is hidden in the network's fast dynamics, which does not exist in the non-frozen STP-RNN (Rahmati et al, 2017). This unstable FP, which disappears after the peak of the simGDPs, enables the emergence of simGDPs by building an amplification threshold (magenta line, Figure 5C) in the initial activity phase of the STP-RNN operating at its rest state.…”

“…A Neural Network Model with Excitatory GABA Accounts for Major Experimental Observations Using a recently established developing recurrent neural network model (RNN) (Rahmati et al, 2017), we investigated whether we can qualitatively reproduce and reveal mechanistic insights into our experimental observations. The network is a mean-field model of activity rates of two populations of glutamatergic (PC) and GABAergic (SOM) cells with short-term synaptic plasticity (STP-RNN; Figure 5A), here parameterized with excitatory GABA and depressing synapses.…”

Neuronal Synchronization Can Control the Energy Efficiency of Inter-Spike Interval Coding

“…However, during neocortical development, the activity of developing neural networks transitions from this highly synchronized mode (with a synchronization level of about 80%) to a relatively sparse mode after the postnatal onset of sensory transduction [44,[87][88][89]. While this so-called sparsification phenomenon is thought to refine sensory coding [58], our findings may provide some additional mechanistic explanation for the actual purpose of sparsification. That is, our resutls ( Figure 7C) suggest that the neuronal communication in immature cortices lacking sensory inputs (e.g., in mice visual cortex during first postnatal week [43,44,90]) may not be reliably energy efficient.…”

“…the portion of neurons participating in the synchronous event) can be modulated by e.g., the features of the external stimulus (e.g. orientation, intensity level, or context), switching attention [40,45,[55][56][57], or the internal state of the network [58]. Nonetheless, to model the synaptic activity with a controlled synchronization level we use a simple procedure for generating spikes and synchronous patterns.…”