Stereo Vision Based Sensory Substitution for the Visually Impaired

Abstract

The development of computer vision based systems dedicated to help visually impaired people to perceive the environment, to orientate and navigate has been the main research subject of many works in the recent years. A significant ensemble of resources has been employed to support the development of sensory substitution devices (SSDs) and electronic travel aids for the rehabilitation of the visually impaired. The Sound of Vision (SoV) project used a comprehensive approach to develop such an SSD, tackling all the challenging aspects that so far restrained the large scale adoption of such systems by the intended audience: Wearability, real-time operation, pervasiveness, usability, cost. This article is set to present the artificial vision based component of the SoV SSD that performs the scene reconstruction and segmentation in outdoor environments. In contrast with the indoor use case, where the system acquires depth input from a structured light camera, in outdoors SoV relies on stereo vision to detect the elements of interest and provide an audio and/or haptic representation of the environment to the user. Our stereo-based method is designed to work with wearable acquisition devices and still provide a real-time, reliable description of the scene in the context of unreliable depth input from the stereo correspondence and of the complex 6 DOF motion of the head-worn camera. We quantitatively evaluate our approach on a custom benchmarking dataset acquired with SoV cameras and provide the highlights of the usability evaluation with visually impaired users.

A Review of Technologies and Techniques for Indoor Navigation Systems for the Visually Impaired

“…Its operation is like time-based methods, but instead of using the distances to the anchor nodes, it checks the angles. There are usually two main methods of obtaining the AoA of a signal [ 52 , 53 ]: Use a matrix of sensors whose locations relative to the center of the node are known and use the difference in ToA of the signal at each sensor to calculate the AoA of the anchor node. Two or more directional antennas are pointed in different directions with overlapping main beams.…”

“…Caraiman et al developed a project called Sound of Vision (SoV) to provide an indoor navigation guide for people with visual impairments [ 52 ]. SoV’s premises were to meet a set of requirements such as being wearable, performing user guide operations in real time, respecting usability issues, and having a low acquisition cost.…”

Visual Echolocation Concept for the Colorophone Sensory Substitution Device Using Virtual Reality

“…Perception of shape, colour and depth are basic functions of the visual system that contribute to orientation and mobility in the surrounding environment. Therefore, many Sensory Substitution Devices (SSDs) are designed with the aim of coding shape [ 1 , 2 ], colour [ 3 , 4 ] and distance [ 3 , 5 , 6 , 7 ] in the form of non-visual stimuli, in order to provide the information for those who cannot access them through vision. When thinking about the navigation and mobility of the visually impaired, some of the information, especially distance, is crucial.…”

Navigation aid for blind persons by visual-to-auditory sensory substitution: A pilot study

“…The SoV uses a combination of stereo camera and infrared sensor, both worn on the forehead, to capture a depth information array of the scene and translate each point of the array into the volume of an auditory output signal [42]. The auditory output of the array happens simultaneously, resulting in a high update rate, which allows for an easier understanding of the 3D shape of the environment.…”