Simulation of visual perception and learning with a retinal prosthesis

Abstract

The nature of artificial vision with a retinal prosthesis, and the degree to which the brain can adapt to the unnatural input from such a device, are poorly understood. Therefore, the development of current and future devices may be aided by theory and simulations that help to infer and understand what patients see. A novel computational framework was developed to predict visual perception and the effect of learning with a subretinal prosthesis. The framework is based on the idea that the central visual system efficiently reconstructs the incident image from the retinal output. To implement this idea, a simulation of the normal responses of the major retinal ganglion cell types was used to deduce the optimal linear reconstruction of the visual stimulus from retinal activity. The result was then used to make inferences about visual experience with simulated retinal activation by a subretinal prosthesis. The inferred visual perception obtained with prosthesis activation was substantially degraded compared to the inferred perception obtained with normal retinal responses, as expected given the limited resolution and lack of cell type specificity of the prosthesis. Consistent with the importance of cell type specificity, reconstruction using only ON cells, and not OFF cells, was substantially more accurate. Finally, when reconstruction was re-optimized for electrical stimulation, simulating learning by the patient, the accuracy of inferred perception with prosthesis stimulation was closer to that of natural vision. The reconstruction approach provides a framework for interpreting patient data in clinical trials, and may be useful for improving prosthesis design.

A computational observer model of spatial contrast sensitivity: Effects of wavefront-based optics, cone mosaic structure, and inference engine

“…As treatments for partial sight restoration become feasible, for example through gene therapy or retinal prostheses, it will be important to understand the degree to which the additional information provided to the nervous system by these technologies support performance. The ability to use simulations to model the information carried by restored representations and understand the upper limits on visual performance available from them should facilitate the design of therapies that can ameliorate partial and full blindness (Cottaris & Elfar, 2005;Jiang, Wandell, & Farrell, 2015;Golden et al, 2018;Beyeler, Boynton, Fine, & Rokem, 2018).…”

What do blind people “see” with retinal prostheses? Observations and qualitative reports of epiretinal implant users

“…While many early-stage research studies have reported that implant-elicited percepts are quite variable in appearance (18,51-54), the narrative of punctate, light-colored phosphenes and the scoreboard model dominated in the literature until only recently [49,50,55]. Indeed while projections have been updated with more nuanced knowledge of the effects of electrical stimulation on different cell types in various stages of degeneration and reorganization [56, 57], the devices and rehabilitation protocols are still designed and built on the assumption that the quality of artificial vision produced depends on the spatial acuity of the array, as determined by electrode size, number and spacing, where each electrode will ideally produce a circumscribed phosphene.…”

Retinal ganglion cells undergo cell type—specific functional changes in a computational model of cone-mediated retinal degeneration

“…We have developed a biophysically inspired in silico model of the cone pathway in the retina that simulates the network-level response to both light and electrical stimulation, and found that simulated cone-mediated retinal degeneration differentially affects ON and OFF RGCs. Existing computational models of retinal degeneration largely focus on RGC activity in the absence of photoreceptor input (e.g., Cottaris and Elfar, 2005 ; Golden et al, 2018 ), but do not consider the global retinal remodeling that may impact the responsiveness of RGCs. To this end, our simulations do not just reproduce commonly reported findings about the changes in RGC activity encountered during retinal degeneration (e.g., hyperactivity, increased electrical thresholds) but also offer testable predictions about the neuroanatomical mechanisms that may underlie altered RGC activity as a function of disease progression.…”

“…For instance, Cottaris and Elfar ( 2005 ) built a model of the healthy retina and removed the cone population without addressing biophysical changes to the inner retina. Golden et al ( 2018 ) simulated degeneration by removing a fraction of simulated neurons, increasing connectivity among the surviving neurons, and increasing the noise level, but did not address the progressive nature of these diseases. Other models stopped at reducing the thickness of different retinal layers (Paknahad et al, 2020 , 2021 ) or hard-coded known physiological changes, such as increased spontaneous activity, into their model (Loizos et al, 2018 ).…”