Deep-Learning-Based Stair Detection Using 3D Point Cloud Data for Preventing Walking Accidents of the Visually Impaired

Matsumura, Hideaki; Premachandra, Chinthaka

doi:10.1109/access.2022.3178154

Cited by 18 publications

(8 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Significant progress has been made in the research of using Convolutional Neural Networks (CNN) for RGB-D salient object detection [19][20][21][22][23][24][25][26][27][28][29][30][31]. For instance, Fan et al [20] utilized two parallel CNNs to independently process RGB and depth images, and then organically fused the outputs of these two networks.…”

Section: B Multi-modal Fusion-based Salient Object Detectionmentioning

confidence: 99%

“…To protect the privacy of bathers, no RGB images were captured using a 2D camera. Matsumura et al [26] provided an accident prevention method using deep learning techniques to notify visually impaired individuals about the presence of height and steps when approaching stairs. The study employed deep learning on generated 3D point cloud data to detect stairs.…”

Section: B Multi-modal Fusion-based Salient Object Detectionmentioning

confidence: 99%

“…Existing salient object detection methods [19][20][21][22][23][24][25][26][27] primarily rely on deep learning technologies, especially Convolutional Neural Networks (CNNs). However, consecutive convolution and pooling operations significantly reduce the size of deep feature maps, which, although able to accurately locate salient objects, cannot finely segment them.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

RGB-D Salient Object Detection Method Based on Multi-Modal Fusion and Contour Guidance

Peng,

Feng,

Zheng

2023

IEEE Access

View full text Add to dashboard Cite

Salient object detection is a critical task in the field of computer vision. However, existing detection methods still face certain challenges, such as the inability to effectively integrate multimodal features and the blurring of detection result boundaries. To address these issues, this paper proposes a novel RGB-D salient object detection method that combines multimodal feature fusion and contour-guiding techniques. Initially, we employ ResNet50 as the backbone network, and by removing its final pooling layer and fully connected layer, we construct a fully convolutional network specifically for feature extraction from RGB images and depth images. Subsequently, we leverage channel attention mechanism and spatial attention mechanism separately to optimize the RGB image features and depth image features. Following this, we design an interactive feature fusion module to blend the optimized features, thereby obtaining the multimodal fusion features. Furthermore, based on the localization ability of high-level fusion features, we constrain the low-level fusion features, eliminate non-salient objects, and generate salient object contour features. Eventually, we use this contour feature to guide the recognition process of salient objects, resulting in salient objects with clear boundaries. Our approach has been validated across seven RGB-D salient object detection datasets. The experimental results indicate an improvement of 0.21% ~1.84% and 0.32% ~1.25% respectively in maxF and S metrics, compared to the best competing methods (CMINet, CIR-Net, and CPFP).

show abstract

Section: B Multi-modal Fusion-based Salient Object Detectionmentioning

confidence: 99%

Section: B Multi-modal Fusion-based Salient Object Detectionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

RGB-D Salient Object Detection Method Based on Multi-Modal Fusion and Contour Guidance

Peng,

Feng,

Zheng

2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Therefore the classification techniques of point clouds have become an attractive field for the researchers and different algorithms have been developed to grasp better knowledge of objects present in the generated point cloud [25]. Eminent researchers have shown recently a deep learning based stair detection method applied on the developed 3D point cloud from RealSense depth camera [26]. A novel segmentation method has been presented for extraction of planar laser scan lines using the technique based on testing a range of residuals against a range of thresholds [27].…”

Section: B Point Cloud Classification Techniquesmentioning

confidence: 99%

Parametric Classification of Furniture From Point Cloud Developed Using Low Cost Trolley Based Laser Scanning System

et al. 2023

View full text Add to dashboard Cite

This paper presents a parametric classification methodology to identify common indoor and outdoor furniture objects present in the 3D Cartesian point cloud of the surveyed environment. For this purpose, a low cost custom made trolley based scanning and surveying system has developed using orthogonal integration of two popular Hokuyo-30LX 2D laser scanners. The developed system has been successfully used to generate 3D point cloud of the environment using Simultaneous Localization and Mapping (SLAM) technique. The instrumentation system of the trolley has been interfaced through Robot Operating System (ROS) for online processing and recording of all sensorial data. While classification of the furniture present in point cloud has been done in offline mode using Random Sampling and Consensus (RANSAC) based parametric segmentation technique. The innovative furniture detection has applied on each scan in order to reduce the region of interest in the developed point cloud. In addition, the validation of the classified furniture objects has been performed using Fuzzy Logic. Multiple indoor and outdoor vicinities have been scanned and modelling results have been found accurate nearer to ground truth. In comparison to available surveying solutions present in the local market, the developed system has been found faster and precise to produce more enhanced structural results with minute details.

show abstract

“…The basic issue of precise location navigation still persists as GPS are not suitable for blind persons due to its minute errors (limitation of precision within 10 meters). The words done on this regard [33,34] were mostly simulation based and no blind persons were involved in the experimentation process.…”

Section: Introductionmentioning

confidence: 99%

A Smart Clothing Approach for Augmenting Mobility of Visually Impaired People

Bhatlawande,

Borse,

Solanke

et al. 2024

IEEE Access

View full text Add to dashboard Cite

As per World Health Organization (WHO), a total of around 2.2 Billion of human population in the world has visual problem to severe or minor extend. In this paper, an assistive system is proposed to augment the mobility of such visually impaired people. This system is realized by using a smart clothing approach for wearable convenience of a subject. In this system, five ultrasonic sensors, a web camera and a global positioning receiver (GPS) are integrated on a wearable jacket. This multisensory arrangement is proposed in order to translate human navigation ability into an electronic system. It employs machine learning for reliable detection of obstacles and navigation path. Proposed system also assists in knowing information about current or nearby location, planning a journey from a reference location to a desired destination, and tracking the navigation path to reach the destination. The functional features of this system are resolved based on continual interactions with the visually impaired subjects, their care takers and rehabilitation professionals. The said system was subjected to clinical evaluations with ten totally blind subjects. Results of this study showed encouraging improvement in mobility performances of subjects as compared to a white cane alone. Amongst the participated subjects, 80% acknowledged usefulness of the proposed system for attending their mobility needs. Total 60% subjects admitted decrease in cognitive efforts, and 80% acknowledged reduction in perceptual efforts because of this system.

show abstract

Deep-Learning-Based Stair Detection Using 3D Point Cloud Data for Preventing Walking Accidents of the Visually Impaired

Cited by 18 publications

References 18 publications

RGB-D Salient Object Detection Method Based on Multi-Modal Fusion and Contour Guidance

RGB-D Salient Object Detection Method Based on Multi-Modal Fusion and Contour Guidance

Parametric Classification of Furniture From Point Cloud Developed Using Low Cost Trolley Based Laser Scanning System

A Smart Clothing Approach for Augmenting Mobility of Visually Impaired People

Contact Info

Product

Resources

About