Retinal Encoding of Natural Scenes

Karamanlis, Dimokratis; Schreyer, Helene Marianne; Gollisch, Tim

doi:10.1146/annurev-vision-100820-114239

Cited by 14 publications

(9 citation statements)

References 146 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The modelling of retinal responses to natural stimuli has advanced our understanding of the complexity of retinal processing in recent years. As suggested in a recent review, it is helpful to consider the contributions of different studies in terms of one of three perspectives on the retinal encoding of natural scenes: The circuit perspective (“how?”), the normative perspective (“why?”), and the coding perspective (“what?”) (69). For example, an in-silico dissection of a CNN model of the retina offered explanations on how the surprisingly complex retinal computations, such as motion reversal, omitted stimulus response, and polarity reversal, emerge from simpler computations within retinal circuits (26, 27).…”

Section: Discussionmentioning

confidence: 99%

A chromatic feature detector in the retina signals visual context changes

Hoefling

Szatko

Behrens

et al. 2022

Preprint

View full text Add to dashboard Cite

The retina transforms patterns of light into visual feature representations supporting behaviour. These representations are distributed across various types of retinal ganglion cells (RGCs), whose spatial and temporal tuning properties have been extensively studied in many model organisms, including the mouse. However, it has been difficult to link the potentially nonlinear retinal transformations of natural visual inputs to specific ethological purposes. Here, we discover a novel selectivity to chromatic contrast in an RGC type that allows the detection of transitions of the horizon across a retinal region. We trained a convolutional neural network (CNN) model on large-scale functional recordings of RGC responses to natural mouse movies, and then used this model to search in silico for stimuli that maximally excite distinct types of RGCs. This procedure predicted centre colour-opponency in transient Suppressed-by-Contrast RGCs (tSbC), a cell type whose function is being debated. We confirmed experimentally that these cells indeed responded very selectively to Green-OFF, UV-ON contrasts, which we found to be characteristic of transitions from ground to sky in the visual scene, as might be elicited by head- or eye-movements across the horizon. Because tSbCs reliably detected these transitions, we suggest a role for this RGC type in providing contextual information (i.e. sky or ground) necessary for the selection of appropriate behavioural responses to other stimuli, such as looming objects. Our work showcases how a combination of experiments with natural stimuli and computational modelling allows discovering novel types of stimulus selectivity and identifying their potential ethological relevance.

show abstract

Section: Discussionmentioning

confidence: 99%

A chromatic feature detector in the retina signals visual context changes

Hoefling

Szatko

Behrens

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Instead, the natural retinal input is dynamically structured by eye and head movements that rapidly shift the retinal image (Land, 2015). Large gaze shifts can induce response transients at fixation onset in neurons at the early stages of the visual system (Miura and Scanziani, 2022; Noda and Adey, 1974), and natural inputs drive nonlinear retinal processing (Heitman et al, 2016; Karamanlis et al, 2022; Shah et al, 2020; Turner and Rieke, 2016; Turner et al, 2018; Yu et al, 2022), which is currently missing from models of efficient coding. Here, we sought to study whether stimulus correlations in gaze-based natural movies are efficiently discarded by the retina.…”

mentioning

confidence: 99%

Natural stimuli drive concerted nonlinear responses in populations of retinal ganglion cells

Karamanlis

Khani

Schreyer

et al. 2023

Preprint

Self Cite

View full text Add to dashboard Cite

The role of the vertebrate retina in early vision is generally described by the efficient coding theory, which predicts that the retina discards spatiotemporal correlations in natural scenes. It is unclear, however, whether the predicted decorrelation in the activity of ganglion cells, the retina's output neurons, holds under gaze shifts, which dominate the natural visual input. We here show that species-specific gaze patterns in natural stimuli can drive strong and correlated spiking responses both within and across distinct types of ganglion cells in marmoset as well as mouse retina. These concerted responses violate efficient coding and signal fixation periods with locally high spatial contrast. Finally, novel model-based analyses of ganglion cell responses to natural stimuli reveal that the observed response correlations follow from nonlinear pooling of ganglion cell inputs. Our results reveal how concerted population activity can surpass efficient coding to detect gaze-related stimulus features.

show abstract

“…Most of our understanding of the retina’s neural code is at a single-cell level based on responses to artificial stimuli, like drifting gratings [20, 68]. It is less well understood how natural stimuli are encoded by the population activity.…”

Section: Resultsmentioning

confidence: 99%

“…This is particularly important, as our gaze only remains fixed for a fraction of a second, before rapid saccadic eye movements induce a new visual scene onto the retina [66,67]. Most of our understanding of the retina's neural code is at a single-cell level based on responses to artificial stimuli, like drifting gratings [20,68]. It is less well understood how natural stimuli are encoded by the population activity.…”

Section: Emergence Of a Relative Spike Latency Code And Stimulus Omis...mentioning

confidence: 99%

Temporal prediction captures retinal spiking responses across animal species

Taylor,

Zenke,

King

et al. 2024

Preprint

View full text Add to dashboard Cite

The retina's role in visual processing has been viewed as two extremes: an efficient compressor of incoming visual stimuli akin to a camera, or as a predictor of future stimuli. Addressing this dichotomy, we developed a biologically-detailed spiking retinal model trained on natural movies under metabolic-like constraints to either encode the present or to predict future scenes. Our findings reveal that when optimized for efficient prediction approximately 100 ms into the future, the model not only captures retina-like receptive fields and their mosaic-like organizations, but also exhibits complex retinal processes such as latency coding, motion anticipation, differential tuning, and stimulus-omission responses. Notably, the predictive model also more accurately predicts the way retinal ganglion cells respond across different animal species to natural images and movies. Our findings demonstrate that the retina is not merely a compressor of visual input, but rather is fundamentally organized to provide the brain with foresight into the visual world.

show abstract

Retinal Encoding of Natural Scenes

Cited by 14 publications

References 146 publications

A chromatic feature detector in the retina signals visual context changes

A chromatic feature detector in the retina signals visual context changes

Natural stimuli drive concerted nonlinear responses in populations of retinal ganglion cells

Temporal prediction captures retinal spiking responses across animal species

Contact Info

Product

Resources

About