A Smartphone-Based Mobility Assistant Using Depth Imaging for Visually Impaired and Blind

Abstract

The means of assisting visually impaired and blind (VIB) people when travelling usually relies on other people. Assistive devices have been developed to assist in blind navigation, but many technologies require users to purchase more devices and they lack flexibility, thus making it inconvenient for VIB users. In this research, we made use of a mobile phone with a depth camera function for obstacle avoidance and object recognition. It includes a mobile application that is controlled using simple voice and gesture controls to assist in navigation. The proposed system gathers depth values from 23 coordinate points that are analyzed to determine whether an obstacle is present in the head area, torso area, or ground area, or is a full body obstacle. In order to provide a reliable warning system, the research detects outdoor objects within a distance of 1.6 m. Subsequently, the object detection function includes a unique interactable feature that enables interaction with the user and the device in finding indoor objects by providing an audio and vibration feedback, and users were able to locate their desired objects more than 80% of the time. In conclusion, a flexible and portable system was developed using a depth camera-enabled mobile phone for use in obstacle detection without the need to purchase additional hardware devices.

An Auditory System Interface for Augmented Accessibility: Empowering the Visually Impaired

“…The author of [14,15] describes a successful mobile application for mobility aid that gives users access to a portable, adaptable navigational assistive device that combines object detection and obstacle detection. It can be operated with gestures and a voicecommand-enabled user interface.…”

Development of Blind Campus Navigation System with Obstacle Detection Device

“…According to the opinion of some visually impaired individuals, their major problem is to locate a place precisely [4]. Assisting the visually impaired and blind persons when moving from one place to another usually relies on other persons [15]. Visual impairment (VI) is a condition of reduced visual function that cannot be improved by the use of glasses, surgery or other medical methods.…”

Outdoor Navigation Assistive System Based on Robust and Real-Time Visual–Auditory Substitution Approach

“…Visual information of the environment is also used to detect and localise nearby obstacles by applying deep-learning methods [ 25 ]. Recently, the improved hardware performance coupled with an efficient deep learning architecture designed for mobile implementation [ 29 , 30 ] allowed for real-time execution with low power consumption. In addition, the semantic information of an object such as this nature improves the understanding of the degree of danger that this object represents.…”