Efficient Sparse Coding in Early Sensory Processing: Lessons from Signal Recovery

Lörincz, András; Palotai, Zsolt; Szirtes, Gábor

doi:10.1371/journal.pcbi.1002372

Cited by 14 publications

(12 citation statements)

References 48 publications

(88 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Work has also been performed on learning receptive field properties and visual models from the statistics of natural image data (Field [59]; van der Schaaf and van Hateren [60]; Olshausen and Field [61]; Rao and Ballard [62]; Simoncelli and Olshausen [63]; Geisler [64]; Hyvärinen et al [65]; Lörincz [66]) and been shown to lead to the formation of similar receptive fields as found in biological vision. The proposed theory of receptive fields can be seen as describing basic physical constraints under which a learning based method for the development of receptive fields will operate and the solutions to which an optimal adaptive system may converge to, if exposed to a sufficiently large and representative set of natural image data (see Figure 17).…”

Section: B Relations To Approaches For Learning Receptive Fields From...mentioning

confidence: 99%

A computational theory of visual receptive fields

2013

View full text Add to dashboard Cite

A receptive field constitutes a region in the visual field where a visual cell or a visual operator responds to visual stimuli. This paper presents a theory for what types of receptive field profiles can be regarded as natural for an idealized vision system, given a set of structural requirements on the first stages of visual processing that reflect symmetry properties of the surrounding world. These symmetry properties include (i) covariance properties under scale changes, affine image deformations, and Galilean transformations of space–time as occur for real-world image data as well as specific requirements of (ii) temporal causality implying that the future cannot be accessed and (iii) a time-recursive updating mechanism of a limited temporal buffer of the past as is necessary for a genuine real-time system. Fundamental structural requirements are also imposed to ensure (iv) mutual consistency and a proper handling of internal representations at different spatial and temporal scales. It is shown how a set of families of idealized receptive field profiles can be derived by necessity regarding spatial, spatio-chromatic, and spatio-temporal receptive fields in terms of Gaussian kernels, Gaussian derivatives, or closely related operators. Such image filters have been successfully used as a basis for expressing a large number of visual operations in computer vision, regarding feature detection, feature classification, motion estimation, object recognition, spatio-temporal recognition, and shape estimation. Hence, the associated so-called scale-space theory constitutes a both theoretically well-founded and general framework for expressing visual operations. There are very close similarities between receptive field profiles predicted from this scale-space theory and receptive field profiles found by cell recordings in biological vision. Among the family of receptive field profiles derived by necessity from the assumptions, idealized models with very good qualitative agreement are obtained for (i) spatial on-center/off-surround and off-center/on-surround receptive fields in the fovea and the LGN, (ii) simple cells with spatial directional preference in V1, (iii) spatio-chromatic double-opponent neurons in V1, (iv) space–time separable spatio-temporal receptive fields in the LGN and V1, and (v) non-separable space–time tilted receptive fields in V1, all within the same unified theory. In addition, the paper presents a more general framework for relating and interpreting these receptive fields conceptually and possibly predicting new receptive field profiles as well as for pre-wiring covariance under scaling, affine, and Galilean transformations into the representations of visual stimuli. This paper describes the basic structure of the necessity results concerning receptive field profiles regarding the mathematical foundation of the theory and outlines how the proposed theory could be used in further studies and modelling of biological vision. It is also shown how receptive field responses can be interpreted physically,...

show abstract

Section: B Relations To Approaches For Learning Receptive Fields From...mentioning

confidence: 99%

A computational theory of visual receptive fields

2013

View full text Add to dashboard Cite

show abstract

“…In previous decades, substantial advances have been made to determine the strategy used by neural systems to work efficiently while saving energy, including optimizing ion channel kinetics (Alle et al, 2009 ; Schmidt-Hieber and Bischofberger, 2010 ), developing a warm body temperature to minimize the energy cost of single action potentials (Yu et al, 2012 ), optimizing the number of channels on single neurons and the number of neurons in neuronal networks (Schreiber et al, 2002 ; Yu and Liu, 2014 ; Yu et al, 2016 ), maintaining a low probability of releasing neurotransmitters at synapses (Levy and Baxter, 2002 ; Harris et al, 2012 ), representing information with sparse spikes (Olshausen and Field, 2004 ; Lorincz et al, 2012 ; Yu et al, 2014 ), optimizing the inter- and intra-regional wiring of the cortex (Mitchison, 1991 ; Chklovskii and Koulakov, 2004 ), arranging functional connectivity among brain regions in the form of a “small world” network (Bassett and Bullmore, 2006 ; Tomasi et al, 2013 ), and other techniques. These studies demonstrate the possibility that a trade-off between energy cost and information processing capacity driven by selective pressure could shape the morphology and physiology of neural systems to optimize for energy efficiency.…”

Section: Introductionmentioning

confidence: 99%

Efficient Coding and Energy Efficiency Are Promoted by Balanced Excitatory and Inhibitory Synaptic Currents in Neuronal Network

Shen

Wang

2018

Front. Cell. Neurosci.

View full text Add to dashboard Cite

Selective pressure may drive neural systems to process as much information as possible with the lowest energy cost. Recent experiment evidence revealed that the ratio between synaptic excitation and inhibition (E/I) in local cortex is generally maintained at a certain value which may influence the efficiency of energy consumption and information transmission of neural networks. To understand this issue deeply, we constructed a typical recurrent Hodgkin-Huxley network model and studied the general principles that governs the relationship among the E/I synaptic current ratio, the energy cost and total amount of information transmission. We observed in such a network that there exists an optimal E/I synaptic current ratio in the network by which the information transmission achieves the maximum with relatively low energy cost. The coding energy efficiency which is defined as the mutual information divided by the energy cost, achieved the maximum with the balanced synaptic current. Although background noise degrades information transmission and imposes an additional energy cost, we find an optimal noise intensity that yields the largest information transmission and energy efficiency at this optimal E/I synaptic transmission ratio. The maximization of energy efficiency also requires a certain part of energy cost associated with spontaneous spiking and synaptic activities. We further proved this finding with analytical solution based on the response function of bistable neurons, and demonstrated that optimal net synaptic currents are capable of maximizing both the mutual information and energy efficiency. These results revealed that the development of E/I synaptic current balance could lead a cortical network to operate at a highly efficient information transmission rate at a relatively low energy cost. The generality of neuronal models and the recurrent network configuration used here suggest that the existence of an optimal E/I cell ratio for highly efficient energy costs and information maximization is a potential principle for cortical circuit networks.SummaryWe conducted numerical simulations and mathematical analysis to examine the energy efficiency of neural information transmission in a recurrent network as a function of the ratio of excitatory and inhibitory synaptic connections. We obtained a general solution showing that there exists an optimal E/I synaptic ratio in a recurrent network at which the information transmission as well as the energy efficiency of this network achieves a global maximum. These results reflect general mechanisms for sensory coding processes, which may give insight into the energy efficiency of neural communication and coding.

show abstract

“…In order to survive, organisms navigating the sensory world need to extract as much information as possible from their environment. The bandwidth of initial sensory processing by peripheral receptors constitutes a critical bottleneck for the nervous system's ability to extract and represent high-dimensional information from the environment (Barlow, 2001;Lorincz et al, 2012). Chemosensation is a particularly acute example.…”

Section: Introductionmentioning

confidence: 99%

Computational algorithms and neural circuitry for compressed sensing in the mammalian main olfactory bulb

Kepple

Cazakoff

Demmer

et al. 2018

Preprint

View full text Add to dashboard Cite

A major challenge for many sensory systems is the representation of stimuli that vary along many dimensions. This problem is particularly acute for chemosensory systems because they require sensitivity to a large number of molecular features. Here we use a combination of computational modeling and in vivo electrophysiological data to propose a solution for this problem in the circuitry of the mammalian main olfactory bulb. We model the input to the olfactory bulb as an array of chemical features that, due to the vast size of chemical feature space, is sparsely occupied. We propose that this sparseness enables compression of the chemical feature array by broadly-tuned odorant receptors. Reconstruction of stimuli is then achieved by a supernumerary network of inhibitory granule cells. The main olfactory bulb may therefore implement a compressed sensing algorithm that presents several advantages. First, we demonstrate that a model of synaptic interactions between the granule cells and the mitral cells that constitute the output of the olfactory bulb, can store a highly efficient representation of odors by competitively selecting a sparse basis set of "expert" granule cells. Second, we further show that this model network can simultaneously learn separable representations of each component of an odor mixture without exposure to those components in isolation. Third, our model is capable of independent and odor-specific adaptation, which could be used by the olfactory system to perform background subtraction or sensitively compare a sample odor with an internal expectation. This model makes specific predictions about the dynamics of granule cell activity during learning. Using in vivo electrophysiological recordings, we corroborate these predictions in an experimental paradigm that stimulates memorization of odorants.

show abstract

Efficient Sparse Coding in Early Sensory Processing: Lessons from Signal Recovery

Cited by 14 publications

References 48 publications

A computational theory of visual receptive fields

A computational theory of visual receptive fields

Efficient Coding and Energy Efficiency Are Promoted by Balanced Excitatory and Inhibitory Synaptic Currents in Neuronal Network

Computational algorithms and neural circuitry for compressed sensing in the mammalian main olfactory bulb

Contact Info

Product

Resources

About