The dynamic neural code of the retina for natural scenes

Abstract

The normal function of the retina is to convey information about natural visual images. It is this visual environment that has driven evolution, and that is clinically relevant. Yet nearly all of our understanding of the neural computations, biological function, and circuit mechanisms of the retina comes in the context of artificially structured stimuli such as flashing spots, moving bars and white noise. It is fundamentally unclear how these artificial stimuli are related to circuit processes engaged under natural stimuli. A key barrier is the lack of methods for analyzing retinal responses to natural images. We addressed both these issues by applying convolutional neural network models (CNNs) to capture retinal responses to natural scenes. We find that CNN models predict natural scene responses with high accuracy, achieving performance close to the fundamental limits of predictability set by intrinsic cellular variability. Furthermore, individual internal units of the model are highly correlated with actual retinal interneuron responses that were recorded separately and never presented to the model during training. Finally, we find that models fit only to natural scenes, but not white noise, reproduce a range of phenomena previously described using distinct artificial stimuli, including frequency doubling, latency encoding, motion anticipation, fast contrast adaptation, synchronized responses to motion reversal and object motion sensitivity. Further examination of the model revealed extremely rapid context dependence of retinal feature sensitivity under natural scenes using an analysis not feasible from direct examination of retinal responses. Overall, these results show that nonlinear retinal processes engaged by artificial stimuli are also engaged in and relevant to natural visual processing, and that CNN models form a powerful and unifying tool to study how sensory circuitry produces computations in a natural context.

Inferring hidden structure in multilayered neural circuits

“…Regarding mechanisms, the spatiotemporal linear filter of the LN model is typically interpreted as mapping onto the aggregate sequential mechanisms of phototransduction, signal filtering and transmission through bipolar and amacrine cell pathways, and summation at the ganglion cell, while the nonlinearity is mapped onto the spiking threshold of ganglion cells. Regarding accuracy, while previous studies have found that these simple models can, for some neurons, capture most of the variance of the responses to low-resolution spatiotemporal white noise [9, 12, 20], they do not describe responses to stimuli with more structure such as natural scenes [13, 30–33]. …”

“…Another common assumption is that the subunit nonlinearities have a particular form, such as quadratic [11, 25] or sigmoidal [44]. Fitting multi-layered models with convolutional filters and fixed nonlinearities has also been successfully used to describe retinal responses to natural scenes [32, 33], although this work maximizes predictive accuracy at the expense of a one-to-one mapping of model components onto retinal circuit elements. Finally, other work focuses on particular ganglion cell types with a small number of inputs [22], constrains the input stimulus to a low-dimensional subspace (such as two halves of the receptive field [45]), or constrains the coefficients of receptive fields to be non-negative [46], thus discarding known properties of the inhibitory surround.…”

Seeing Natural Images through the Eye of a Fly with Remote Focusing Two-Photon Microscopy

“…Natural scenes have been used for reverse correlation analysis, which generally yielded receptive fields comparable to the ones obtained with white noise stimuli (Ringach et al, 2002;Theunissen et al, 2001;Touryan et al, 2005;Vance et al, 2016;Willmore et al, 2010). On the other hand, a recent study highlights the dynamic nonlinearities during natural scene stimulation in salamander retinal ganglion cells (Maheswaranathan et al, 2019), where white-noise-derived models are not able to explain their data well. Overall, whereas many response properties of cells can be probed with artificial stimuli, the use of natural stimuli can further our understanding of a dynamic neural circuit that has evolved in natural conditions.…”

Metaheuristic Optimisation Algorithms for Tuning a Bioinspired Retinal Model

“…proposed a set of computational retinal microcircuits that can be used as basic building blocks for the modelling of different retina mechanisms [30]. Recently, deep convolutional neural networks were shown to capture retinal responses to natural scenes, with results that were close to the variability of the cellular response range [31,32], as well as multitask recurrent neural networks that provided the necessary flexibility to model complex neuronal computations [33].…”