Smart Glass System Using Deep Learning for the Blind and Visually Impaired

Abstract

Individuals suffering from visual impairments and blindness encounter difficulties in moving independently and overcoming various problems in their routine lives. As a solution, artificial intelligence and computer vision approaches facilitate blind and visually impaired (BVI) people in fulfilling their primary activities without much dependency on other people. Smart glasses are a potential assistive technology for BVI people to aid in individual travel and provide social comfort and safety. However, practically, the BVI are unable move alone, particularly in dark scenes and at night. In this study we propose a smart glass system for BVI people, employing computer vision techniques and deep learning models, audio feedback, and tactile graphics to facilitate independent movement in a night-time environment. The system is divided into four models: a low-light image enhancement model, an object recognition and audio feedback model, a salient object detection model, and a text-to-speech and tactile graphics generation model. Thus, this system was developed to assist in the following manner: (1) enhancing the contrast of images under low-light conditions employing a two-branch exposure-fusion network; (2) guiding users with audio feedback using a transformer encoder–decoder object detection model that can recognize 133 categories of sound, such as people, animals, cars, etc., and (3) accessing visual information using salient object extraction, text recognition, and refreshable tactile display. We evaluated the performance of the system and achieved competitive performance on the challenging Low-Light and ExDark datasets.

Improved Real-Time Fire Warning System Based on Advanced Technologies for Visually Impaired People

“…We introduced a client–server scheme containing a smartphone and smart glasses as the client and an AI server to execute the image analysis [ 25 ]. The client section includes smart glasses and a smartphone that transfers information through Bluetooth and a home surveillance camera that records consistently.…”

Design principles of an open-source language modeling microservice package for AAC text-entry applications

“…There is a growing body of literature either solely devoted to or mentioning the use of microservices architectures in healthcare, in particular in health information systems (HIS) (Mavrogiorgou et al, 2021), mobility (Melis et al, 2016;Rendulich et al, 2019;Mukhiddinov and Cho, 2021), physiotherapy (Caporuscio et al, 2017;Postolache et al, 2019), and elderly patients care (Dobre et al, 2021;Panchea et al, 2021). Furthermore, there is a growing awareness of the importance of flexible software architectures in assistive technologies as the Web becomes even more ubiquitous (Yesilada and Harper, 2019).…”

Current and future perspectives on smart glasses

“…Artificial intelligence and computer vision approaches facilitate individuals with blindness and visual impairment in accomplishing primary activities without depending on others. Smart glasses are a prospective assistive technology for these individuals to aid in individual activities and provide social ease and safety [27]. Augmented-reality smart glasses are an innovative technology in the initial stage of expansion with a prospective future and market share ahead.…”