Interpreting the retinal neural code for natural scenes: From computations to neurons

Maheswaranathan, Niru; McIntosh, Lane; Tanaka, Hidenori; Grant, Satchel; Kastner, David B.; Melander, Joshua B.; Nayebi, Aran; Brezovec, Luke E.; Wang, Julia; Ganguli, Surya; Baccus, Stephen A.

doi:10.1016/j.neuron.2023.06.007

Cited by 26 publications

(35 citation statements)

References 56 publications

(78 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This includes finding certain units tuned to the direction and orientation of moving gratings, as has similarly been reported in the retina [74]; units that exclusively fire to high-spatial-frequency gratings only if they move, like Y-type cells in the retina [20]; units tuned for the differential motion of objects versus their background [25]; units tuned for the anticipation of moving objects [24]; and units tuned for stimulus omission within a temporal sequence of flashing lights [22,23]. Certain complex retinal phenomena (like omission and anticipation responses) have been shown to emerge in non-spiking encoding models of the retina [75,76]. These phenomena emerge as a consequence of fitting models to retinal responses, whereas our normative approach rather examines whether these phenomena can be explained as a consequence of underlying principles like compression or prediction.…”

Section: Discussionmentioning

confidence: 99%

Temporal prediction captures retinal spiking responses across animal species

Taylor,

Zenke,

King

et al. 2024

Preprint

View full text Add to dashboard Cite

The retina's role in visual processing has been viewed as two extremes: an efficient compressor of incoming visual stimuli akin to a camera, or as a predictor of future stimuli. Addressing this dichotomy, we developed a biologically-detailed spiking retinal model trained on natural movies under metabolic-like constraints to either encode the present or to predict future scenes. Our findings reveal that when optimized for efficient prediction approximately 100 ms into the future, the model not only captures retina-like receptive fields and their mosaic-like organizations, but also exhibits complex retinal processes such as latency coding, motion anticipation, differential tuning, and stimulus-omission responses. Notably, the predictive model also more accurately predicts the way retinal ganglion cells respond across different animal species to natural images and movies. Our findings demonstrate that the retina is not merely a compressor of visual input, but rather is fundamentally organized to provide the brain with foresight into the visual world.

show abstract

Section: Discussionmentioning

confidence: 99%

Temporal prediction captures retinal spiking responses across animal species

Taylor,

Zenke,

King

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Popular computational methods of subunit inference in recent years have been linear-nonlinear cascade models (Maheswaranathan et al 2018; Real et al 2017), approaches of convolutional neural network models (Maheswaranathan et al 2023; McIntosh et al 2016; Tanaka et al 2019), and methods of statistical inference (Liu et al 2017; Shah et al 2020). Linear-nonlinear-linear-nonlinear (LNLN) models consist of a layer of linear filters of the spatial or spatiotemporal subunits with nonlinear transfer functions that additively converge into another nonlinear transfer function to produce a firing rate or spiking probability.…”

Section: Discussionmentioning

confidence: 99%

“…Training can be done with natural images instead of artificial stimuli like white noise or flickering bars. For models trained with ganglion cell data from the salamander retina, the obtained convolutional filters shared properties with actual bipolar and amacrine cells (Maheswaranathan et al 2023; Tanaka et al 2019), although it remains unclear to what extent the model architecture resembles the actual neural circuitry. Spike-triggered clustering (Shah et al 2020), on the other hand, recovers subunits on stimulus correlations much like STNMF.…”

Section: Discussionmentioning

confidence: 99%

“…The methods include data-driven fitting of cascade networks that model subunits as linear filters with a nonlinear transfer function converging onto a ganglion cell (Freeman et al 2015; Maheswaranathan et al 2018; McFarland et al 2013). Similarly, convolutional neural networks, which offer a comparable model of signal convergence with potentially deeper layering have been used to capture how local processing inside the receptive field of a ganglion cell shapes the cell’s visual responses (Maheswaranathan et al 2023; McIntosh et al 2016; Tanaka et al 2019). On the other side of the spectrum, there are statistical analyses of a cell’s stimulus-response relationship, such as spike-triggered clustering (Shah et al 2020) and spike-triggered non-negative matrix factorization (STNMF; Liu et al 2017).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Accelerated spike-triggered non-negative matrix factorization reveals coordinated ganglion cell subunit mosaics in the primate retina

Zapp,

Khani,

Schreyer

et al. 2024

Preprint

View full text Add to dashboard Cite

A standard circuit motif in sensory systems is the pooling of sensory information from an upstream neuronal layer. A downstream neuron thereby collects signals across different locations in stimulus space, which together compose the neuron's receptive field. In addition, nonlinear transformations in the signal transfer between the layers give rise to functional subunits inside the receptive field. For ganglion cells in the vertebrate retina, for example, receptive field subunits are thought to correspond to presynaptic bipolar cells. Identifying the number and locations of subunits from the stimulus-response relationship of a recorded ganglion cell has been an ongoing challenge in order to characterize the retina's functional circuitry and to build computational models that capture nonlinear signal pooling. Here we present a novel version of spike-triggered non-negative matrix factorization (STNMF), which can extract localized subunits in ganglion-cell receptive fields from recorded spiking responses under spatiotemporal white-noise stimulation. The method provides a more than 100-fold speed increase compared to a previous implementation, which can be harnessed for systematic screening of hyperparameters, such as sparsity regularization. We demonstrate the power and flexibility of this approach by analyzing populations of ganglion cells from salamander and primate retina. We find that subunits of midget as well as parasol ganglion cells in the marmoset retina form separate mosaics that tile visual space. Moreover, subunit mosaics show alignment with each other for ON and OFF midget as well as for ON and OFF parasol cells, indicating a spatial coordination of ON and OFF signals at the bipolar-cell level. Thus, STNMF can reveal organizational principles of signal transmission between successive neural layers, which are not easily accessible by other means.

show abstract

“…An additional useful aspect of these models is that they can be analyzed to determine the set of model interneurons that generated any particular response (Tanaka et al, 2019) (Maheswaranathan et al, 2023). This analysis, termed an attribution analysis, derives from approaches of interpretable machine learning (Sundararajan et al, 2017), where the goal is to gain insight into how the properties of the model generated particular outputs.…”

Section: Introductionmentioning

confidence: 99%

Classification and analysis of retinal interneurons by computational structure under natural scenes

Lee,

Kim,

Baccus

2024

Preprint

Self Cite

View full text Add to dashboard Cite

Inhibitory neurons are diverse across the brain, but for the visual system we lack the ability to functionally classify these neurons under complex natural stimuli. Here we take the approach of classifying retinal amacrine cell responses to natural scenes using optical recording and an interpretable neural network model. We fit mouse amacrine cell responses to a two-layer convolutional neural network model of a class shown previously to accurately capture salamander ganglion cell responses to natural scenes. Using an approach from interpretable machine learning, we determined for each stimulus the model interneurons that generated each amacrine response, analogous to the set of bipolar cells that target the amacrine population.From this analysis we clustered amacrine cells not by their natural scene responses, but by the model presynaptic neurons that constructed those responses, conservatively finding approximately seven groups by this approach. By analyzing the set of model presynaptic input neurons for each amacrine cluster, we find that distributed rather than dedicated inputs generate natural scene responses for different amacrine cell types. Additional analyses revealed distinct transient and sustained modes exhibited by the network during the response to simple flashes. These results give insight into the computational structure of how the diverse amacrine cell population responds to natural scenes, and generate multiple quantitative hypotheses for how synaptic inputs generate those responses.

show abstract

Interpreting the retinal neural code for natural scenes: From computations to neurons

Cited by 26 publications

References 56 publications

Temporal prediction captures retinal spiking responses across animal species

Temporal prediction captures retinal spiking responses across animal species

Accelerated spike-triggered non-negative matrix factorization reveals coordinated ganglion cell subunit mosaics in the primate retina

Classification and analysis of retinal interneurons by computational structure under natural scenes

Contact Info

Product

Resources

About