Deep Learning Based Stair Detection and Statistical Image Filtering for Autonomous Stair Climbing

Patil, Unmesh; Gujarathi, Aniket; Kulkarni, Akshay; Jain, Aman; Malke, Lokeshkumar; Tekade, Radhika; Paigwar, Kartik; Chaturvedi, Pradyumn

doi:10.1109/irc.2019.00031

Cited by 29 publications

(18 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The effectiveness of the proposed model is evaluated with existing staircase recognition and debris detection approach. The staircase detection approach is compared with the Unmesh et al [ 26 ], Munoz et al [ 41 ] and Wang et al [ 14 ] schemes. Here, the Munoz et al [ 41 ] and Wang [ 14 ] staircase detection schemes are employed using fusion of Hough transform and SVM classifier.…”

Section: Results and Discussionmentioning

confidence: 99%

“…Here, the Munoz et al [ 41 ] and Wang [ 14 ] staircase detection schemes are employed using fusion of Hough transform and SVM classifier. In [ 26 ] the authors use the tiny-YOLOv3 CNN framework for recognizing the staircase, and Hough Transform and Statistical filtering based image processing pipeline for steps detection. Table 5 indicates the comparison results of the proposed scheme with the above-mentioned three methods.…”

Section: Results and Discussionmentioning

confidence: 99%

“…The comparative study was performed based on the detection accuracy of each model. YOLO based CNN frameworks are faster than SSD MobileNet but have a low accuracy [ 26 ]. SSD-MobileNet has shown a better trade off between accuracy and computation time.…”

Section: Results and Discussionmentioning

confidence: 99%

“…Instead of traditional computer vision based techniques, learning based environmental perception (e.g., Deep Learning) is a new paradigm. The deep learning neural network layers can learn features to distinguish and determine objects at high accuracy and can be used in various vision tasks such as outdoor environmental perception for autonomous vehicles [ 18 , 19 , 20 ], cleaning and maintenance robot vision pipeline [ 2 , 21 , 22 , 23 , 24 ], mobile robot place recognition and mapping tasks [ 25 , 26 , 27 , 28 ]. While these state-of-the-art deep learning algorithms have achieved impressive results for environment perception design, there are some shortcomings such as false detection, which can be due to objects with similar features.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

sTetro-Deep Learning Powered Staircase Cleaning and Maintenance Reconfigurable Robot

Ramalingam

Mohan

Balakrishnan

et al. 2021

Sensors

View full text Add to dashboard Cite

Staircase cleaning is a crucial and time-consuming task for maintenance of multistory apartments and commercial buildings. There are many commercially available autonomous cleaning robots in the market for building maintenance, but few of them are designed for staircase cleaning. A key challenge for automating staircase cleaning robots involves the design of Environmental Perception Systems (EPS), which assist the robot in determining and navigating staircases. This system also recognizes obstacles and debris for safe navigation and efficient cleaning while climbing the staircase. This work proposes an operational framework leveraging the vision based EPS for the modular re-configurable maintenance robot, called sTetro. The proposed system uses an SSD MobileNet real-time object detection model to recognize staircases, obstacles and debris. Furthermore, the model filters out false detection of staircases by fusion of depth information through the use of a MobileNet and SVM. The system uses a contour detection algorithm to localize the first step of the staircase and depth clustering scheme for obstacle and debris localization. The framework has been deployed on the sTetro robot using the Jetson Nano hardware from NVIDIA and tested with multistory staircases. The experimental results show that the entire framework takes an average of 310 ms to run and achieves an accuracy of 94.32% for staircase recognition tasks and 93.81% accuracy for obstacle and debris detection tasks during real operation of the robot.

show abstract

Section: Results and Discussionmentioning

confidence: 99%

Section: Results and Discussionmentioning

confidence: 99%

Section: Results and Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

sTetro-Deep Learning Powered Staircase Cleaning and Maintenance Reconfigurable Robot

Ramalingam

Mohan

Balakrishnan

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…Additionally, some state of the art staircase localization systems have been implemented using RGB images, deep learning and point clouds. Deep learning methods to detect staircases use neuronal networks such as YOLOv3-tiny [8,9], YOLOv4 [8] and SSD [10,11] or implement a stair step detection using YOLOv3-tiny [12] and staircase segmentation using AlbuNet [13]. The use of CNNs to detect an object in an image allows further processing in the detected area or region of interest (ROI).…”

Section: Introductionmentioning

confidence: 99%

Staircase Detection, Characterization and Approach Pipeline for Search and Rescue Robots

et al. 2021

View full text Add to dashboard Cite

Currently, most rescue robots are mainly teleoperated and integrate some level of autonomy to reduce the operator’s workload, allowing them to focus on the primary mission tasks. One of the main causes of mission failure are human errors and increasing the robot’s autonomy can increase the probability of success. For this reason, in this work, a stair detection and characterization pipeline is presented. The pipeline is tested on a differential drive robot using the ROS middleware, YOLOv4-tiny and a region growing based clustering algorithm. The pipeline’s staircase detector was implemented using the Neural Compute Engines (NCEs) of the OpenCV AI Kit with Depth (OAK-D) RGB-D camera, which allowed the implementation using the robot’s computer without a GPU and, thus, could be implemented in similar robots to increase autonomy. Furthermore, by using this pipeline we were able to implement a Fuzzy controller that allows the robot to align itself, autonomously, with the staircase. Our work can be used in different robots running the ROS middleware and can increase autonomy, allowing the operator to focus on the primary mission tasks. Furthermore, due to the design of the pipeline, it can be used with different types of RGB-D cameras, including those that generate noisy point clouds from low disparity depth images.

show abstract

Augmented Reality (AR) Application Superimposing the Falling Risks of Older Adults in Residential Settings and Coping Strategies: Building an Image-Based Scene Detection Model

Miura,

Uchiyama,

Imaeda

et al. 2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Deep Learning Based Stair Detection and Statistical Image Filtering for Autonomous Stair Climbing

Cited by 29 publications

References 13 publications

sTetro-Deep Learning Powered Staircase Cleaning and Maintenance Reconfigurable Robot

sTetro-Deep Learning Powered Staircase Cleaning and Maintenance Reconfigurable Robot

Staircase Detection, Characterization and Approach Pipeline for Search and Rescue Robots

Augmented Reality (AR) Application Superimposing the Falling Risks of Older Adults in Residential Settings and Coping Strategies: Building an Image-Based Scene Detection Model

Contact Info

Product

Resources

About