Indoor/Outdoor Semantic Segmentation Using Deep Learning for Visually Impaired Wheelchair Users

Abstract

Electrical Powered Wheelchair (EPW) users may find navigation through indoor and outdoor environments a significant challenge due to their disabilities. Moreover, they may suffer from near-sightedness or cognitive problems that limit their driving experience. Developing a system that can help EPW users to navigate safely by providing visual feedback and further assistance when needed can have a significant impact on the user's wellbeing. This paper presents computer vision systems based on deep learning, with an architecture based on residual blocks that can semantically segment high-resolution images. The systems are modified versions of DeepLab version 3 plus that can process high-resolution input images. Besides, they can simultaneously process images from indoor and outdoor environments, which is challenging due to the difference in data distribution and context. The proposed systems replace the base network with a smaller one and modify the encoder-decoder architecture. Nevertheless, they produce high-quality outputs with fast inference speed compared to the systems with deeper base networks. Two datasets are used to train the semantic segmentation systems: an indoor application-based dataset that has been collected and annotated manually and an outdoor dataset to cover both environments. The user can toggle between the two individual systems depending on the situation. Moreover, we introduce shared systems that automatically use a specific semantic segmentation system depending on the pixels' confidence scores. The annotated output scene is presented to the EPW user, which can aid with the user's independent navigation. Stateof-the-art semantic segmentation techniques are discussed and compared. Results show the ability of the proposed systems to detect objects with sharp edges and high accuracy for indoor and outdoor environments. The developed systems are deployed on a GPU based board and then integrated on an EPW for practical usage and evaluation. The used indoor dataset is made publicly available online.

Sistema de Baixo Custo para Auxílio de Detecção de Obstáculos para Pessoas com Deficiência Visual

“…No trabalho [14], é apresentado o sistema DEEP-SEE, que utiliza algoritmos de visão computacional e redes neurais convolucionais profundas (CNNs) para detecção, rastreamento e reconhecimento de objetos em tempo real. Por fim, em [11], oferece-se suporte à locomoção de pessoas com deficiência visual e cadeirantes, utilizando deep learning em ambientes tanto internos quanto externos.…”

“…Nos últimos anos, as soluções acadêmicas para essa área têm empregado processamento de imagem por meio de IA, demandando considerável capacidade computacional, como destacado em [4] e [11]. Isso resulta em custos elevados para suas aplicações, devido às especificações necessárias.…”

Obstacle Detection and Distance Estimation for Visually Impaired People

“…However, most of these innovations, like the use of smartphones for object detection or the deployment of non-wearable sensors, have limitations in terms of portability and real-time functionality. For instance, [1] demonstrated a smartphone-based system for object detection, which, while effective, necessitates handheld operation and lacks distance estimation capabilities. Similarly, [2] explored the use of ultrasonic sensors attached to canes, offering some distance information but at the expense of user convenience and range of detection.…”

A Dataset for Temporal Semantic Segmentation Dedicated to Smart Mobility of Wheelchairs on Sidewalks

“…In [ 64 ], an attention model captures the spatial consistency of low-level features in the temporal dimension. In [ 65 ], the DeepLabV3+ architecture is modified to get fast inference time on a GPU, and trained with a home-made dataset [ 7 ]. The system is then embedded on an EW for indoor and outdoor environments.…”