Computer Vision for the Visually Impaired: the Sound of Vision System

Caraiman, Simona; Morar, Anca; Owczarek, M.; Burlacu, Adrian; Rzeszotarski, Dariusz; Botezatu, Nicolae; Herghelegiu, Paul; Moldoveanu, Florica; Strumiłło, Paweł; Moldoveanu, Alin

doi:10.1109/iccvw.2017.175

Cited by 79 publications

(64 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The basis of our developed crosswalk detection is Choi et al's idea of using a horizontal mean filter [7]. We combined the horizontal with a vertical filter and defined a decision process.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Camera-based On-Road Detections for the Visually Impaired

Jakob¹,

Tick²

2020

ACTA POLYTECH HUNG

View full text Add to dashboard Cite

Herein, we research the possibilities of assisting visually impaired pedestrians moving in traffic situations by using camera-based detection of relevant objects in their immediate surroundings. Therefore, we use and adapt algorithms from the field of driver assistance. We present a road background segmentation based on watersheds, whose results are used as input for the presented crosswalk and lane detection algorithms. The crosswalk detection is based on the application of two 1D mean filters and the lane detection on local computations of the EDF (Edge Distribution Function). In our evaluation, the described algorithms achieved good hit rates of 99.87% (road segmentation), 98.64% (crosswalk detection), and 97.89% (lane detection).

show abstract

Section: Discussionmentioning

confidence: 99%

“…[29]), Sensory Substitution Devices (SSD) offer a comprehensive perception of the environment by transferring the sense of sight to a different sense, e.g. hearing [7] or tactile sense [1].…”

Section: Camera-based Assistance For the Visually Impairedmentioning

confidence: 99%

Camera-based On-Road Detections for the Visually Impaired

Jakob¹,

Tick²

2020

ACTA POLYTECH HUNG

View full text Add to dashboard Cite

show abstract

“…Such assistive devices require the cooperation of researchers from different fields, such as medicine, smart electronics, computer science, and engineering. So far, as a result of this interdisciplinary cooperation, several designs and components of wearable camera-enabled systems for VCP have been proposed [2][3][4][5]. Such systems incorporate sensors, such as cameras, ultrasonic sensors, laser distance sensors, inertial measurement units, microphones, and GPS, which enable the user identify his/her position in an area of interest (i.e., outdoor environment, hospital, museum, archeological site, etc.…”

Section: Introductionmentioning

confidence: 99%

“…In [22], a wearable camera system was proposed, capable of identifying walkable spaces, planning a safe motion trajectory in space, recognizing and localizing certain types of objects, as well as providing haptic-feedback to the user through vibrations. A system named Sound of Vision was presented in [5], aiming to provide the users with a 3D representation of the surrounding environment, conveyed by means of hearing and tactile senses. The system comprised an RGB-depth (RGB-D) sensor and an inertial measurement unit (IMU) to track the head/camera orientation.…”

mentioning

confidence: 99%

Uncertainty-Aware Visual Perception System for Outdoor Navigation of the Visually Challenged

Dimas

Diamantis

Kalozoumis

et al. 2020

Sensors

View full text Add to dashboard Cite

Every day, visually challenged people (VCP) face mobility restrictions and accessibility limitations. A short walk to a nearby destination, which for other individuals is taken for granted, becomes a challenge. To tackle this problem, we propose a novel visual perception system for outdoor navigation that can be evolved into an everyday visual aid for VCP. The proposed methodology is integrated in a wearable visual perception system (VPS). The proposed approach efficiently incorporates deep learning, object recognition models, along with an obstacle detection methodology based on human eye fixation prediction using Generative Adversarial Networks. An uncertainty-aware modeling of the obstacle risk assessment and spatial localization has been employed, following a fuzzy logic approach, for robust obstacle detection. The above combination can translate the position and the type of detected obstacles into descriptive linguistic expressions, allowing the users to easily understand their location in the environment and avoid them. The performance and capabilities of the proposed method are investigated in the context of safe navigation of VCP in outdoor environments of cultural interest through obstacle recognition and detection. Additionally, a comparison between the proposed system and relevant state-of-the-art systems for the safe navigation of VCP, focused on design and user-requirements satisfaction, is performed.

show abstract

“…Seeing AI has numerous features such as reading texts, recognizing friends, and describing the surrounding scene [23]. Caraiman et al [6] proposed the Sound of Vision System, a wearable device that reconstructs the sensed environment and segmentation into objects of interest and conveying the necessary information through auditory and tactile representation. The device is able to identify elements such as signs, texts, doors, and pedestrian crossing.…”

Section: Introductionmentioning

confidence: 99%

Street Crossing Aid Using Light-Weight CNNs for the Visually Impaired

Lee

Kim

2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

In this paper, we address an issue that the visually impaired commonly face while crossing intersections and propose a solution that takes form as a mobile application. The application utilizes a deep learning convolutional neural network model, LytNetV2, to output necessary information that the visually impaired may lack when without human companions or guide-dogs. A prototype of the application runs on iOS devices of versions 11 or above. It is designed for comprehensiveness, concision, accuracy, and computational efficiency through delivering the two most important pieces of information, pedestrian traffic light color and direction, required to cross the road in real-time. Furthermore, it is specifically aimed to support those facing financial burden as the solution takes the form of a free mobile application. Through the modification and utilization of key principles in MobileNetV3 such as depthwise seperable convolutions and squeeze-excite layers, the deep neural network model achieves a classification accuracy of 96% and average angle error of 6.15 • , while running at a frame rate of 16.34 frames per second. Additionally, the model is trained as an image classifier, allowing for a faster and more accurate model. The network is able to outperform other methods such as object detection and non-deep learning algorithms in both accuracy and thoroughness. The information is delivered through both auditory signals and vibrations, and it has been tested on seven visually impaired and has received above satisfactory responses.

show abstract

Computer Vision for the Visually Impaired: the Sound of Vision System

Cited by 79 publications

References 36 publications

Camera-based On-Road Detections for the Visually Impaired

Camera-based On-Road Detections for the Visually Impaired

Uncertainty-Aware Visual Perception System for Outdoor Navigation of the Visually Challenged

Street Crossing Aid Using Light-Weight CNNs for the Visually Impaired

Contact Info

Product

Resources

About