Non-visual-cueing-based sensing and understanding of nearby entities in aided navigation.

Abstract

Exploring unfamiliar environments is a challenging task in which additionally, unsighted individuals frequently fail to gain perception of obstacles and make serendipitous discoveries. This is because the mental depiction of the context is drastically lessened due to the absence of visual information. It is still not clear in neuroscience, whether stimuli elicited by visual cueing can be replicated by other senses (cross-model transfer). In the practice, however, everyone recognizes a key, whether it is felt in a pocket or seen on a table. We present a context-aware aid system for the blind that merges three levels of assistance enhancing the intelligibility of the nearby entities: an exploration module to help gain awareness of the surrounding context, an alerting method for warning the user when a stumble is likely, and, finally, a recognition engine that retrieves natural targets previously learned. Practical experiences with our system show that in the absence of visual cueing, the audio and haptic trajectory playback coupled with computer-vision methods is a promising approach to depict dynamic information of the immediate environment.

See ColOr: an extended sensory substitution device for the visually impaired

“…In the global module the image is made accessible on a tactile tablet that makes it possible for the user to compare different points and explore the scene in a broader context [35], [40] ( Figure 3). A user may rapidly scan an image sliding one or more fingers on this tablet.…”

“…Once the system launches a warning, the user is expected to suspend the navigation not to bump into an obstacle. This allows the blind persons finding a safe, clear path to advance through [40], [41]. Roughly, when potential obstacle in the video presenting a distance below one meter continues to approach over a given number of frames, the user must be alerted (Figure 4, right).…”