Face identity recognition in simulated prosthetic vision is poorer than previously reported and can be improved by caricaturing

Abstract

The visual prosthesis (or "bionic eye") has become a reality but provides a low resolution view of the world. Simulating prosthetic vision in normal-vision observers, previous studies report good face recognition ability using tasks that allow recognition to be achieved on the basis of information that survives low resolution well, including basic category (sex, age) and extra-face information (hairstyle, glasses). Here, we test within-category individuation for face-only information (e.g., distinguishing between multiple Caucasian young men with hair covered). Under these conditions, recognition was poor (although above chance) even for a simulated 40×40 array with all phosphene elements assumed functional, a resolution above the upper end of current-generation prosthetic implants. This indicates that a significant challenge is to develop methods to improve face identity recognition. Inspired by "bionic ear" improvements achieved by altering signal input to match high-level perceptual (speech) requirements, we test a high-level perceptual enhancement of face images, namely face caricaturing (exaggerating identity information away from an average face). Results show caricaturing improved identity recognition in memory and/or perception (degree by which two faces look dissimilar) down to a resolution of 32×32 with 30% phosphene dropout. Findings imply caricaturing may offer benefits for patients at resolutions realistic for some current-generation or in-development implants.

Caricaturing can improve facial expression recognition in low-resolution images and age-related macular degeneration

“…This high-level approach has the added benefit that such techniques do not depend on the specifics of retinal damage or exact visual appearance in any individual patient or any given disorder. Caricaturing has the potential to lead to practical benefits in patients a range of low-vision disorders and, indeed, even in patients without functioning eyes at all (e.g., via prosthetic implants in LGN or cortical area V1; Irons et al, 2017).…”

Semantic and structural image segmentation for prosthetic vision

“…Based on previous studies with simulated prosthetic vision [9,71,72], we approximate the phosphenes as grayscale circular dots with a Gaussian luminance profile -each phosphene has maximum intensity at the center and gradually decays to the periphery, following a Gaussian function-. The intensity of a phosphene is directly extracted from the intensity of the same region in the processed image.…”

Caricaturing faces to improve identity recognition in low vision simulations: How effective is current-generation automatic assignment of landmark points?

“…Low-resolution vision occurs in many eye conditions, including age-related macular degeneration [ 1 – 3 ] and prosthetic vision (the ‘bionic eye ’ [ 4 – 7 ]) ( Fig 1 ). In low vision, the ability to recognise individual faces is poor (e.g., poor ability to tell apart a set of young adult Caucasian males with short hair) [ 1 – 3 , 8 , 9 ]. Poor identity recognition is associated with significant difficulties in real-world social interaction [ 10 , 11 ], and thus it is important to develop techniques that can improve face identity perception.…”