Testing Vision Is Not Testing For Vision

Abstract

Visual prostheses aim to restore, at least to some extent, vision that leads to the type of perception available for sighted patients. Their effectiveness is almost always evaluated using clinical tests of vision. Clinical vision tests are designed to measure the limits of parameters of a functioning visual system. I argue here that these tests are rarely suited to determine the ability of prosthetic devices and other therapies to restore vision. This paper describes and explains many limitations of these evaluations. Prosthetic vision testing often makes use of multiple-alternative forced-choice (MAFC) procedures. Although these paradigms are suitable for many studies, they are frequently problematic in vision restoration evaluation. Two main types of problems are identified: (1) where nuisance variables provide spurious cues that can be learned in repeated training, which is common in prosthetic vision, and thus defeat the purpose of the test; and (2) even though a test is properly designed and performed, it may not actually measure what the researchers believe, and thus the interpretation of results is wrong. Examples for both types of problems are presented. Additional problems arise from confounding factors in the administration of tests are pointed as limitations of current device evaluation. For example, head tracing of magnified objects enlarged to compensate for the system's low resolution, in distinction from the scanning head (camera) movements with which users of prosthetic devices expand the limited field of view. Because of these problems, the ability to perform satisfactorily on the clinical tests is necessary but insufficient to prove vision restoration, therefore, additional tests are needed. I propose some directions to pursue in such testing. Translational Relevance Numerous prosthetic devices are being developed and introduced to the market. Proving the utility of these devices is crucial for regulatory and even for post market acceptance, which so far has largely failed, in my opinion. Potential reasons for the failures despite success in regulatory testing and directions for designing improved testing are provided. It is hoped that improved testing will guide improved designs of future prosthetic systems and other vision restoration approaches.

Sequential epiretinal stimulation improves discrimination in simple shape discrimination tasks only

“…Some tasks can be performed by scanning the field-of-view of the camera (Caspi and Zivotofsky 2015; Hallum and Dakin 2021) or the region-of-interest using eye movements (Caspi et al 2018). However, scanning limits the performance of the system (Peli 2020), making it critical to develop stimulation paradigms that better enable patterned sight.…”

Retinal Implantation of Electronic Vision Prostheses to Treat Retinitis Pigmentosa: A Systematic Review

“…Since submitting this systematic review to Translational Vision Science & Technology , two noteworthy Perspectives have been published. 56 , 59 The first proposes guidelines for performing and reporting vision tests in participants implanted with a prosthetic device. 56 Several of these guidelines address issues (i.e., risks of bias) similar to those that we have raised above.…”

“…The second Perspective articulates the pitfalls of attempting to test for vision by using methods designed to measure the limits of a visual system that is assumed to function (normally or partially). 59 Therein, Peli identifies problems with several tests currently in use to assess vision restoration, arguing that the field's progress is contingent on the use of appropriate tests, free of bias, to guide the design of new and better devices.…”

Retinal implantation of electronic vision prostheses to treat retinitis pigmentosa: A systematic review

“…Since submitting this systematic review to Translational Vision Science & Technology (on 15th January, 2020), two noteworthy Perspectives have been published 56,59 . The first proposes guidelines for performing and reporting vision tests in participants implanted with a prosthetic device 56 .…”

“…For example, Bailey and colleagues (subsection Visual Acuity) clearly articulate the need for comparable pre- and post-implantation measurements when assessing visual acuity. The second Perspective articulates the pitfalls of attempting to test for vision by using methods design to measure the limits of a visual system that is assumed to function (normally or partially) 59 . Therein, Peli identifies problems with several tests currently in use to assess vision restoration, arguing that the field’s progress is contingent on the use of appropriate tests, free of bias, to guide the design of new and better devices.…”