Modeling Inhibitory Interneurons in Efficient Sensory Coding Models

Abstract

There is still much unknown regarding the computational role of inhibitory cells in the sensory cortex. While modeling studies could potentially shed light on the critical role played by inhibition in cortical computation, there is a gap between the simplicity of many models of sensory coding and the biological complexity of the inhibitory subpopulation. In particular, many models do not respect that inhibition must be implemented in a separate subpopulation, with those inhibitory interneurons having a diversity of tuning properties and characteristic E/I cell ratios. In this study we demonstrate a computational framework for implementing inhibition in dynamical systems models that better respects these biophysical observations about inhibitory interneurons. The main approach leverages recent work related to decomposing matrices into low-rank and sparse components via convex optimization, and explicitly exploits the fact that models and input statistics often have low-dimensional structure that can be exploited for efficient implementations. While this approach is applicable to a wide range of sensory coding models (including a family of models based on Bayesian inference in a linear generative model), for concreteness we demonstrate the approach on a network implementing sparse coding. We show that the resulting implementation stays faithful to the original coding goals while using inhibitory interneurons that are much more biophysically plausible.

Precise spatial spatial memory in local random networks

“…Similarly, distance-dependent topologies [35] have been implemented in previous models, including the seminal work on continuous neural attractors [25], yet we are aware of only two related studies that link sparse, shortrange (1D nearest-neighbor) connections formally to the localization of firing-rate excitations [36,37]. As we do, both respect Dale's Principle [38] for the signs of synaptic connections only indirectly [39] and explore random weights. While it may be interesting to explore the spectra of our {J ij } in the context of Anderson localization or the notion of "spatially structured" disorder developed in [37], a more obvious generalization of our model would be to relax the hard-threshold cutoff condition to a connection probability.…”

Constrained inference in sparse coding reproduces contextual effects and predicts laminar neural dynamics

“…Moreover, a form of code based on sparseness has many potential benefits for 55 neural systems, being energy efficient Niven and Laughlin (2008), increasing storage capacity in associative 56 memories Baum et al (1988); Charles et al (2014) and making the structure of natural signals explicit and 57 easier to read out at subsequent level of processing Olshausen and Field (2004). Particularly noteworthy is 58 the fact that these statistical models can be reformulated as dynamical systems Rozell et al (2008), where 59 processing units can be identified with real neurons having a temporal dynamics that can be implemented 60 with various degrees of biophysical plausibility: using local learning rules Zylberberg et al (2011), spiking 61 neurons Hu et al (2012); Shapero et al (2013) and even employing distinct classes of inhibitory neurons 62 King et al (2013); Zhu and Rozell (2015). In summary, sparse coding models nicely explain fundamental 63…”

“…given the neurobiological constraints on anatomy and neuronal dynamics. In this regard, the neural 392 implementation proposed by Rozell et al (2008) provided a significant advance, since it can explain a range 393 of contextual effects Zhu and Rozell (2013) with a neural population dynamics that requires only synaptic 394 summation and can also be extended to obey Dale's law Zhu and Rozell (2015). But this model still 395 presents a fundamental, conceptual difference to visual cortex: there are no interactions between neurons 396…”

Biologically plausible learning in a deep recurrent spiking network

“…successfully reproduced simple cell responses in visual cortex [44,45]. 36 It was repeatedly shown that networks with spiking neurons can realize SCNCs [43,[46][47][48][49][50][51][52][53][54][55][56][57][58]. It is, however, not 37 clear how to combine sparse coding circuits to build large recurrent networks with many layers that have the 38 potential to explain neuronal responses and have competitive computational performance.…”

“…There are several detailed models for such circuits that employ spiking neurons [47,48,56]. For the present 529 work, however, we skip detailed modeling of the circuits and replace them by a minimal model -the inference 530 population (IP) -that mimics the neuronal dynamics leading to sparse efficient coding with each impinging 531 spike [58].…”