Iterative Design and Prototyping of Computer Vision Mediated Remote Sighted Assistance

Abstract

Remote sighted assistance (RSA) is an emerging navigational aid for people with visual impairments (PVI). Using scenario-based design to illustrate our ideas, we developed a prototype showcasing potential applications for computer vision to support RSA interactions. We reviewed the prototype demonstrating real-world navigation scenarios with an RSA expert, and then iteratively refined the prototype based on feedback. We reviewed the refined prototype with 12 RSA professionals to evaluate the desirability and feasibility of the prototyped computer vision concepts. The RSA expert and professionals were engaged by, and reacted insightfully and constructively to the proposed design ideas. We discuss what we learned about key resources, goals, and challenges of the RSA prosthetic practice through our iterative prototype review, as well as implications for the design of RSA systems and the integration of computer vision technologies into RSA.

Feasibility of Interactive 3D Map for Remote Sighted Assistance

“…Towards this end, we propose to design a new RSA prototype for assessment. According to prior work [39], we identified several objectives for designing the new RSA service:…”

“…Aira agents emphasize the importance of the live video feed and consider it as the "lifeline of information". Although web-based information (e.g., maps and satellite images) is useful, live video and audio feeds from PVI's mobile devices are the only resource of information that is real-time and guaranteed to be accurate and up-to-date [39]. The basic video chat function is common and can be executed with off-the-shelf tools.…”

“…• Interactive 3D maps. To address the lack of maps in indoor navigation, the RSA agents expect to learn the environmental context of the users from 3D maps in an interactive manner similar to on a web mapping platform (e.g., Google Maps) they have employed in existing RSA services for outdoor navigation [39]. The 3D maps also need to allow agents to change the scale and view via common interactions (e.g., zooming, panning, and rotating) to find both general and detailed information.…”

“…However, the weaker GPS signal strength and low accuracy render these apps unreliable in the indoor environment. Informed by prior study [39], continuously updating the user's position and orientation on a 3D map is desired by professional RSA agents for indoor navigation. To simplify the system, we prefer to realize real-time localization only using standalone mobile devices (phones or tablets) without extra environmental infrastructure (e.g., Bluetooth beacon [42], RFID tags [48]).…”

“…We evaluated the desirability of CV-mediated RSA in our prior work [39]. We developed a low-fidelity prototype showcasing potential applications for CV to support RSA interactions and reviewed it with professional and trained RSA agents.…”