Image Processing for a High-Resolution Optoelectronic Retinal Prosthesis

Abstract

Abstract-In an effort to restore visual perception in retinal diseases such as age-related macular degeneration or retinitis pigmentosa, a design was recently presented for a high-resolution optoelectronic retinal prosthesis having thousands of electrodes. This system requires real-time image processing fast enough to convert a video stream of images into electrical stimulus patterns that can be properly interpreted by the brain. Here, we present image-processing and tracking algorithms for a subretinal implant designed to stimulate the second neuron in the visual pathway, bypassing the degenerated first synaptic layer. For this task, we have developed and implemented: 1) A tracking algorithm that determines the implant's position in each frame. 2) Image cropping outside of the implant boundaries. 3) A geometrical transformation that distorts the image appropriate to the geometry of the fovea. 4) Spatio-temporal image filtering to reproduce the visual processing normally occurring in photoceptors and at the photoreceptor-bipolar cell synapse. 5) Conversion of the filtered visual information into a pattern of electrical current. Methods to accelerate real-time transformations include the exploitation of data redundancy in the time domain, and the use of precomputed lookup tables that are adjustable to retinal physiology and allow flexible control of stimulation parameters. A software implementation of these algorithms processes natural visual scenes with sufficient speed for real-time operation. This computationally efficient algorithm resembles, in some aspects, biological strategies of efficient coding in the retina and could provide a refresh rate higher than fifty frames per second on our system.

Holographic optogenetic stimulation of patterned neuronal activity for vision restoration

“…The projected phase waveform from the SLM can also potentially be used for cancelling corneal aberrations 38 . However, the practical engineering of a high-acuity prosthetic system is highly non-trivial and will require the integration of real-time image-processing for emulating the ganglion-cell encoding of the visual stimuli 39,15 (which generally includes a diversity of response types 40 ), high-resolution eyetracking 39 and real-time hologram computations. Major challenges also remain on the optogenetic probe engineering front-including testing of strategies for safe and effective cellular transfections 7 , and the exploration of favourable red-shifted alternatives to ChR2 (whose excitation peak is effectively absorbed by pigments of the macula lutea in humans 41 ).…”

Chapter 7- Restoring Vision to the Blind: Advancements in Vision Aids for the Visually Impaired

“…In addition, calculation and delivery of the ''natural retinal code-like'' trains of pulses for direct activation of retinal ganglion cells (RGCs) will require eye tracking to monitor movements of the visual scene on the retina. Eye tracking also allows more advanced image processing, including radial stretch and local magnification on the fovea (Asher, Segal, Baccus, Yaroslavsky, & Palanker, 2007), as described in the section Algorithmic Developments below.…”

“…Eye tracking also enables remapping of the retinal images to properly account for radial spread of the cells near the fovea (Asher et al, 2007) for restoration of the central vision, as illustrated in Figure 7.10. Similarly, it can be used for dynamic magnification of the parts of the image corresponding to the PRL, creating an effect of the magnifying glass following the eye's direction of gaze, as illustrated in Figure 7.11.…”

Current and future prospects for optoelectronic retinal prostheses

“…The system also allows location-dependent image processing. 15 More recently, 14 the use of a laser diode for enhanced brightness of the image-projection system has been described. An additional benefit of the photovoltaic version of the system is that pixels operate independently and they can be placed in different locations within the subretinal space.…”