Contextual Integration in Cortical and Convolutional Neural Networks

Iyer, Ramakrishnan; Hu, Brian; Mihalaş, Ştefan

doi:10.3389/fncom.2020.00031

Cited by 9 publications

(16 citation statements)

References 69 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Model of visual processing in the static context. To study optimal context integration in the static condition (where the visual input is static images), we take as a starting point a model proposed by Iyer et al in [26] where model neurons respond to a patch in the visual space -the classical receptive field -but this response is modulated by a larger region of space -the extra-classical receptive field. The extra-classical receptive field contribution is determined by nearby local receptive fields providing indirect input from a larger area of visual space ( Fig.…”

Section: Theoretical Models Of Processing Visual Information In Statimentioning

confidence: 99%

“…There are multiple possible mappings from the probabilistic framework to the neurobiological circuit [26], but the current correspondence is straightforward and yields successful predictions from data, such as like-to like connectivity, as detailed below. When a pair of features is frequently co-occurring, weights between neurons preferential for these features are strong and positive (Fig.…”

Section: Theoretical Models Of Processing Visual Information In Statimentioning

confidence: 99%

“…2c), and weights are proportional to co-occurrence probabilities of neighboring features that are also separated by a time window ∆t. This is a direct generalization of the model in [26] to the time domain, and includes synaptic delay as a biologically motivated constraint. The extended model can capture how local circuit connectivity is shaped by spatio-temporal correlations across receptive fields and across time windows characteristic of biological processes like synaptic delay.…”

Section: Theoretical Models Of Processing Visual Information In Statimentioning

confidence: 99%

“…We study this circuit using a model in which the contextual information is stored in the lateral connections between neurons [26]. Each neuron receives information about the visual scene from feedforward connections (which can be arbitrary in this model), and complements this with surround information provided by nearby neurons.…”

Section: Introductionmentioning

confidence: 99%

“…Outline of paper In section 2.1, we first detail a model introduced in [26] that describes neuronal connections and firing rates of a circuit adapted to static visual scenes (images). We next extend this model to the case of circuits adapted to moving visual scenes (videos).…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Single circuit in V1 capable of switching contexts during movement using VIP population as a switch

Voina

Recanatesi

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

As animals adapt to their environments, their brains are tasked with processing stimuli in different sensory contexts. Whether these computations are context dependent or independent, they are all implemented in the same neural tissue. A crucial question is what neural architectures can respond flexibly to a range of stimulus conditions and switch between them. This is a particular case of flexible architecture that permits multiple related computations within a single circuit.Here, we address this question in the specific case of the visual system circuitry, focusing on context integration, defined as the integration of feedforward and surround information across visual space. We show that a biologically inspired microcircuit with multiple inhibitory cell types can switch between visual processing of the static context and the moving context. In our model, the VIP population acts as the switch and modulates the visual circuit through a disinhibitory motif. Moreover, the VIP population is efficient, requiring only a relatively small number of neurons to switch contexts. This circuit eliminates noise in videos by using appropriate lateral connections for contextual spatio-temporal surround modulation, having superior denoising performance compared to circuits where only one context is learned. Our findings shed light on a minimally complex architecture that is capable of switching between two naturalistic contexts using few switching units.Author SummaryThe brain processes information at all times and much of that information is context-dependent. The visual system presents an important example: processing is ongoing, but the context changes dramatically when an animal is still vs. running. How is context-dependent information processing achieved? We take inspiration from recent neurophysiology studies on the role of distinct cell types in primary visual cortex (V1).We find that relatively few “switching units” — akin to the VIP neuron type in V1 in that they turn on and off in the running vs. still context and have connections to and from the main population — is sufficient to drive context dependent image processing. We demonstrate this in a model of feature integration, and in a test of image denoising. The underlying circuit architecture illustrates a concrete computational role for the multiple cell types under increasing study across the brain, and may inspire more flexible neurally inspired computing architectures.

show abstract

Section: Theoretical Models Of Processing Visual Information In Statimentioning

confidence: 99%

Section: Theoretical Models Of Processing Visual Information In Statimentioning

confidence: 99%

Section: Theoretical Models Of Processing Visual Information In Statimentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Single circuit in V1 capable of switching contexts during movement using VIP population as a switch

Voina

Recanatesi

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Cascaded normalizations for spatial integration in the primary visual cortex of primates

Yang²,

Dai³

et al. 2022

Cell Reports

View full text Add to dashboard Cite

Modelling the role of contour integration in visual inference

Khan

Wong

Tripp

2022

Preprint

View full text Add to dashboard Cite

Under difficult viewing conditions, the brain's visual system uses a variety of recurrent modulatory mechanisms to augment feed-forward processing. One resulting phenomenon is contour integration, which occurs in the primary visual (V1) cortex and strengthens neural responses to edges if they belong to a larger smooth contour. Computational models have contributed to an understanding of the circuit mechanisms of contour integration, but less is known about its role in visual perception. To address this gap, we embedded a biologically grounded model of contour integration in a task-driven artificial neural network, and trained it using a gradient-descent variant. We used this model to explore how brain-like contour integration may be optimized for high-level visual objectives as well as its potential roles in perception. When the model was trained to detect contours in a background of random edges, a task commonly used to examine contour integration in the brain, it closely mirrored the brain in terms of behavior, neural responses, and lateral connection patterns. When trained on natural images, the model enhanced weaker contours and distinguished whether two points lay on the same vs. different contours. The model learnt robust features that generalized well to out-of-training-distribution stimuli. Surprisingly, and in contrast with the synthetic task, a parameter-matched control network without recurrence performed the same or better than the model on the natural-image tasks. Thus a contour integration mechanism is not essential to perform these more naturalistic contour-related tasks. Finally, the best performance in all tasks was achieved by a modified contour integration model that did not distinguish between excitatory and inhibitory neurons.

show abstract

Contextual Integration in Cortical and Convolutional Neural Networks

Cited by 9 publications

References 69 publications

Single circuit in V1 capable of switching contexts during movement using VIP population as a switch

Single circuit in V1 capable of switching contexts during movement using VIP population as a switch

Cascaded normalizations for spatial integration in the primary visual cortex of primates

Modelling the role of contour integration in visual inference

Contact Info

Product

Resources

About