A Wearable Assistive Technology for the Visually Impaired with Door Knob Detection and Real-Time Feedback for Hand-to-Handle Manipulation

Niu, Liang; Qian, Cheng; Rizzo, John-Ross; Hudson, Todd E.; Li, Zi‐Chen; Enright, Shane; Sperling, Eliot; Conti, Kyle; Wong, Edward K.; Fang, Yi

doi:10.1109/iccvw.2017.177

Cited by 20 publications

(21 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A direct comparison with systems already reported in the literature targeting the assistance of the visually impaired is not evident since most of them miss to report their mAPs [29,30,32]. Other object recognition works [44,68,69] for different applications report comparable mAPs between 70% and 90% exploring other deep learning approaches, such as SSD (Single Shot MultiBox Detector), YOLO (You Only Look Once), and R-FCN (Region-based Fully Convolutional Networks) with the use of other image databases for training, such as the PASCAL VOC (Visual Object Classes), SUN (Scene UNderstanding), and the KITTI (Karlsruhe Institute of Technology and Toyota Technological Institute) one.…”

Section: Results Discussionmentioning

confidence: 93%

“…Few systems have targeted VI people needs; Niu et al described, in [29], a wearable system that detects doorknobs and human hands to help the blind people locate and use doors. Panchal et al presented, in [30], a new approach for recognizing text from scene images and to convert it into speech so that it can assist VI people.…”

Section: State Of the Art On Innovative Assistive Technology Devices mentioning

confidence: 99%

See 1 more Smart Citation

Solar-Powered Deep Learning-Based Recognition System of Daily Used Objects and Human Faces for Assistance of the Visually Impaired

et al. 2020

View full text Add to dashboard Cite

This paper introduces a novel low-cost solar-powered wearable assistive technology (AT) device, whose aim is to provide continuous, real-time object recognition to ease the finding of the objects for visually impaired (VI) people in daily life. The system consists of three major components: a miniature low-cost camera, a system on module (SoM) computing unit, and an ultrasonic sensor. The first is worn on the user’s eyeglasses and acquires real-time video of the nearby space. The second is worn as a belt and runs deep learning-based methods and spatial algorithms which process the video coming from the camera performing objects’ detection and recognition. The third assists on positioning the objects found in the surrounding space. The developed device provides audible descriptive sentences as feedback to the user involving the objects recognized and their position referenced to the user gaze. After a proper power consumption analysis, a wearable solar harvesting system, integrated with the developed AT device, has been designed and tested to extend the energy autonomy in the different operating modes and scenarios. Experimental results obtained with the developed low-cost AT device have demonstrated an accurate and reliable real-time object identification with an 86% correct recognition rate and 215 ms average time interval (in case of high-speed SoM operating mode) for the image processing. The proposed system is capable of recognizing the 91 objects offered by the Microsoft Common Objects in Context (COCO) dataset plus several custom objects and human faces. In addition, a simple and scalable methodology for using image datasets and training of Convolutional Neural Networks (CNNs) is introduced to add objects to the system and increase its repertory. It is also demonstrated that comprehensive trainings involving 100 images per targeted object achieve 89% recognition rates, while fast trainings with only 12 images achieve acceptable recognition rates of 55%.

show abstract

Section: Results Discussionmentioning

confidence: 93%

Section: State Of the Art On Innovative Assistive Technology Devices mentioning

confidence: 99%

Solar-Powered Deep Learning-Based Recognition System of Daily Used Objects and Human Faces for Assistance of the Visually Impaired

et al. 2020

View full text Add to dashboard Cite

show abstract

“…A significant component of what supports spatiocognitive activities performed without vision is touch. Obvious applications of touch to environmental awareness and spatial cognition include detecting and identifying objects and localizing key features in the environment (eg, doorknobs, railings, and so on) 9 . However, touch is also an important input for directly supporting safe and efficient navigation.…”

Section: Perspectivementioning

confidence: 99%

“…VIS 4 ION remedies some of the cane's shortcomings, and further augments the ability of BVI persons to both maintain balance and to localize objects in their environment 23,24 . The system also provides robust networked features, which expands computational power through connectivity 9,25‐28 …”

Section: Perspectivementioning

confidence: 99%

COVID‐19 and Visual Disability: Can't Look and Now Don't Touch

Rizzo

Beheshti

Fang

et al. 2021

PM&R

Self Cite

View full text Add to dashboard Cite

The coronavirus disease 19 (COVID-19) pandemic has created cataclysmic repercussions in virtually every facet of life and has had profound effects on the practice of medicine. This is particularly true for providers who treat disability. Although all disabilities are unquestionably challenged in undue ways, this perspective is meant to draw special attention to those with visual impairment. COVID-19 is extremely contagious and has spread globally with unprecedented rapidity. The best current countermeasures include personal protective equipment (PPE), social distancing, and minimizing or avoiding touch or contact with surfaces and/or objects that may be contaminated with viral particles, all of which pose unique challenges for those with low or no vision. Co-authors Rizzo and Giudice, themselves visually impaired, are researchers who are investigating creative innovation to combat the untoward consequences of visual impairment, However, this situation transcends their professional interests, as it has directly affected their lives and the lives of other blind individuals close to them. This essay builds on the combination of their personal experiences and research expertise to motivate the current problem and pose some viable solutions.When you cannot see what is around you, touch becomes the primary mode of both exploring and interacting with the environment. We rely on touch to support many tasks throughout the day, whether it be the movement of the keys as we type on our computer, the warmth and heft we feel as we pick up our morning mug of coffee, or the texture of our clothes. However, for blind and visually impaired (BVI) people, the sense of touch and use of haptics (ie, information that is perceived through active touch) transcends these "normal" uses of this modality. For this community, touch perception supports many of

show abstract

“…These components can be seamlessly integrated into wearables that feature both haptic and audio outputs (human-machine interfaces), such as bone conduction headsets with synthetic speech outputs or vibrating belts or bracelets, such as the VIS 4 ION system (Visually Impaired Smart Service System for Spatial Intelligence and Onboard Navigation). This platform provides real-time situational and obstacle awareness in one's immediate environment, allowing individuals to travel more safely in three-dimensional (3D) space and pays particular attention to low-body, mid-body, and highbody/head hazards [59][60][61][62].…”

Section: Englandmentioning

confidence: 99%

The global crisis of visual impairment: an emerging global health priority requiring urgent action

Rizzo

Beheshti

Hudson

et al. 2020

Disability and Rehabilitation: Assistive Technology

Self Cite

View full text Add to dashboard Cite

A Wearable Assistive Technology for the Visually Impaired with Door Knob Detection and Real-Time Feedback for Hand-to-Handle Manipulation

Cited by 20 publications

References 13 publications

Solar-Powered Deep Learning-Based Recognition System of Daily Used Objects and Human Faces for Assistance of the Visually Impaired

Solar-Powered Deep Learning-Based Recognition System of Daily Used Objects and Human Faces for Assistance of the Visually Impaired

COVID‐19 and Visual Disability: Can't Look and Now Don't Touch

The global crisis of visual impairment: an emerging global health priority requiring urgent action

Contact Info

Product

Resources

About