AIGuide: Augmented Reality Hand Guidance in a Visual Prosthetic

Abstract

Locating and grasping objects is a critical task in people’s daily lives. For people with visual impairments, this task can be a daily struggle. The support of augmented reality frameworks in smartphones can overcome the limitations of current object detection applications designed for people with visual impairments. We present AIGuide, a self-contained smartphone application that leverages augmented reality technology to help users locate and pick up objects around them. We conducted a user study to investigate the effectiveness of AIGuide in a visual prosthetic for providing guidance; compare it to other assistive technology form factors; investigate the use of multimodal feedback, and provide feedback about the overall experience. We gathered performance data and participants’ reactions and analyzed videos to understand users’ interactions with the nonvisual smartphone user interface. Our results show that AIGuide is a promising technology to help people with visual impairments locate and acquire objects in their daily routine. The benefits of AIGuide may be enhanced with appropriate interaction design.

Inclusive AR/VR: accessibility barriers for immersive technologies

“…For users with visual impairments, research has illustrated how immersive technologies can be used as visual aids to support environmental awareness and promote sensory substitution [45]. Furthermore, work has focused on novel interaction methods for expanding users' spatial awareness [46,47,48] and for developing novel user interaction techniques which combine object localisation and spatial audio [49,50] and echolocation [51]. Haptic interactions has also been used with immersive technology for users with visual impairments in novel interfaces to support sensory substitution [52,53,54].…”