A Vision-Based Wayfinding System for Visually Impaired People Using Situation Awareness and Activity-Based Instructions

Abstract

A significant challenge faced by visually impaired people is ‘wayfinding’, which is the ability to find one’s way to a destination in an unfamiliar environment. This study develops a novel wayfinding system for smartphones that can automatically recognize the situation and scene objects in real time. Through analyzing streaming images, the proposed system first classifies the current situation of a user in terms of their location. Next, based on the current situation, only the necessary context objects are found and interpreted using computer vision techniques. It estimates the motions of the user with two inertial sensors and records the trajectories of the user toward the destination, which are also used as a guide for the return route after reaching the destination. To efficiently convey the recognized results using an auditory interface, activity-based instructions are generated that guide the user in a series of movements along a route. To assess the effectiveness of the proposed system, experiments were conducted in several indoor environments: the sit in which the situation awareness accuracy was 90% and the object detection false alarm rate was 0.016. In addition, our field test results demonstrate that users can locate their paths with an accuracy of 97%.

Feasibility of Interactive 3D Map for Remote Sighted Assistance

“…A major focus on CV-based approach is how to make visual information more accessible through recognizing objects [73], obstacles [74], color-codes or landmark (e.g., storefronts [43]), or through processing of tags such as barcodes [75], QR codes [76], and RFID [77]. Extending this focus, researchers have proposed indoor positioning and navigation systems [78,47,79]. However, Saha et al [43] concluded that for a deployable level of accuracy, using CV techniques alone is not sufficient yet.…”

Augmented Reality Glasses as an Orientation and Mobility Aid for People with Low Vision: a Feasibility Study of Experiences and Requirements

“…These aids convey information to users via different modalities: auditory [1,15,29,82,86] and vibrotactile [4,52,69,74]. Audio aids provide information with either verbal warnings and instructions [3,19,58,88] or sonification [7,36,43,61] or both [45]. For example, NavCog3 used BLE beacons for localisation and provide turn-by-turn instructions and descriptions of nearby landmarks and points of interest via bone-conducting headphone [81].…”

Wayfinding Tools for People With Visual Impairments in Real-World Settings: A Literature Review of Recent Studies

“…From a cognitive load perspective, it is not surprising that realtime route information notably increases the rate of success among participants who are visually impaired or blind (Ko and Kim, 2017;Bai et al, 2018;Balata et al, 2018;Giudice et al, 2019). When information is provided in real-time, inclusion of landmarkbased information (Balata et al, 2018), egocentric directions (Giudice et al, 2019), and tailored guidance are preferred by travelers with visual impairments.…”

“…A majority of the studies had preview or training time to learn the wayfinding technology. When technology is being used, additional time for training participants in use of the technology is required (Kalia et al, 2010;Ko and Kim, 2017). The interconnections between the person, the technology, the environment and the systemic support deserves additional attention.…”