Designing a Virtual Environment to Evaluate Multimodal Sensors for Assisting the Visually Impaired

Khoo, Wai L.; Seidel, Eric L.; Zhu, Zhigang

doi:10.1007/978-3-642-31534-3_84

Cited by 6 publications

(8 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a proof of concept, we collected data from four blind subjects with varying ages of visual impairment to assess whether congenitally, early blind, and late blind individuals may benefit more from the BrainPort device than sighted individuals. Due to the restrictions and immobility of the BrainPort Workstation used in our studies, a virtual environment was utilized so that blind subjects could sit in front of a desk to navigate through the virtual environment (Khoo et al, 2012(Khoo et al, , 2013(Khoo et al, , 2015.…”

Section: Methodsmentioning

confidence: 99%

Shape Discrimination Using the Tongue: Implications for a Visual-to-Tactile Sensory Substitution Device

et al. 2016

Self Cite

View full text Add to dashboard Cite

Sensory substitution devices have the potential to provide individuals with visual impairments with more information about their environments, which may help them recognize objects and achieve more independence in their daily lives. However, many of these devices may require extensive training and might be limited in the amount of information that they can convey. We tested the effectiveness and assessed some of the limitations of the BrainPort device, which provides stimulation through a 20 × 20 electrode grid array on the tongue. Across five experiments, including one with blind individuals, we found that subjects were unable to accurately discriminate between simple shapes as well as different line orientations that were briefly presented on the tongue, even after 300 trials of practice with the device. These experiments indicate that such a minimal training regimen with the BrainPort is not sufficient for object recognition, raising serious concerns about the usability of this device without extensive training.

show abstract

Section: Methodsmentioning

confidence: 99%

Shape Discrimination Using the Tongue: Implications for a Visual-to-Tactile Sensory Substitution Device

et al. 2016

Self Cite

View full text Add to dashboard Cite

show abstract

“…The other line of research has focused on augmentation of audio or visual information with tactile information. For example, Apostolopoulos et al [1] and Khoo et al [11] used the vibration function of smart phones to provide tactile information in parallel with auditory information for navigation for blind and low vision users.…”

Section: Tactile Feedbackmentioning

confidence: 99%

Enhancing caption accessibility through simultaneous multimodal information

Kushalnagar

Behm

Stanislow

et al. 2014

Proceedings of the 16th International ACM SIGACCESS Conference on Computers &Amp; Accessibility - ASSETS '14

View full text Add to dashboard Cite

Captions (subtitles) for television and movies have greatly enhanced accessibility for Deaf and hard of hearing (DHH) consumers who do not understand the audio, but can otherwise follow by reading the captions. However, these captions fail to fully convey auditory information, due to simultaneous delivery of aural and visual content, and lack of standardization in representing non-speech information.Viewers cannot simultaneously watch the movie scenes and read the visual captions; instead they have to switch between the two and inevitably lose information and context in watching the movies. In contrast, hearing viewers can simultaneously listen to the audio and watch the scenes.Most auditory non-speech information (NSI) is not easily represented by words, e.g., the description of a ring tone, or the sound of something falling.We enhance captions with tactile and visual-tactile feedback. For the former, we transform auditory NSI into its equivalent tactile representation and convey it simultaneously with the captions. For the latter, we visually identify the location of the NSI. This approach can benefit DHH viewers by conveying more aural content to the viewer's visual and tactile senses simultaneously than visual-only captions alone. We conducted a study, which compared DHH viewer responses between video with captions, tactile captions, and visual-tactile captions. The viewers significantly benefited from visual-tactile and tactile captions. Figure 1: Intertitle-Scene temporal and spatial separation: The movie briefly narrates what will happen in the scene, and then displays the scene.

show abstract

“…Figure 11 Brainport (Khoo et al, 2012) Unlike invasive retina implants, Brainport (Figure 11) from Wicab 8 is a tongue-based electrical stimulation device that conveys brightness contrast of a scene in front of the user through a 20 x 20 electrode array pressed against the tongue. A camera is mounted on a pair of glasses that captures videos, which is then sent to a base unit that processes the data.…”

Section: Brainport (System K)mentioning

confidence: 99%

Multimodal and alternative perception for the visually impaired: a survey

Khoo

Zhu

2016

Journal of Assistive Technologies

Self Cite

View full text Add to dashboard Cite

Purpose This paper provides an overview of navigational assistive technologies with various sensor modalities and alternative perception approaches for visually impaired people. It also examines the input and output of each technology, and provides a comparison between systems. Design/methodology/approach The contributing authors along with their students thoroughly read and reviewed the referenced papers while under the guidance of domain experts and users evaluating each paper/technology based on a set of metrics adapted from universal and system design. Findings After analyzing 13 multimodal assistive technologies, we found that the most popular sensors are optical, infrared, and ultrasonic. Similarly, the most popular actuators are audio and haptic. Furthermore, most systems use a combination of these sensors and actuators. Some systems are niche, while others strive to be universal. Research limitations/implications This paper serves as a starting-point for further research in benchmarking multimodal assistive technologies for the visually impaired and to eventually cultivate better assistive technologies for all. Social Implications Based on 2012 World Health Organization, there are 39 million blind people. This paper will have an insight of what kind of assistive technologies are available to the visually impaired people, whether in market or research lab. Originality/value This paper provides a comparison across diverse visual assistive technologies. This is valuable to those who are developing assistive technologies and want to be aware of what is available as well their pros and cons, and the study of human-computer interfaces.

show abstract

Designing a Virtual Environment to Evaluate Multimodal Sensors for Assisting the Visually Impaired

Cited by 6 publications

References 8 publications

Shape Discrimination Using the Tongue: Implications for a Visual-to-Tactile Sensory Substitution Device

Shape Discrimination Using the Tongue: Implications for a Visual-to-Tactile Sensory Substitution Device

Enhancing caption accessibility through simultaneous multimodal information

Multimodal and alternative perception for the visually impaired: a survey

Contact Info

Product

Resources

About