Image-Based Scene Recognition for Robot Navigation Considering Traversable Plants and Its Manual Annotation-Free Training

Matsuzaki, Shigemichi; Masuzawa, Hiroaki; Miura, Jun

doi:10.1109/access.2022.3141594

Cited by 14 publications

(13 citation statements)

References 49 publications

(58 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although their model performs with high accuracy on a platform with workstationlevel computing power, 3D point cloud data is often very expensive to both collect and compute with, and such data types may not be best suited for low-power mobile platforms, where computing power is extremely limited. The authors in [25] used semantic segmentation as a component to help their robot navigate in greenhouse environments with different types of plants. In their model based on ESPNetv2 [26], the semantic segmentation is used for generic object detection and another algorithm is used for detecting the pixel-level traversability for their navigation.…”

Section: B Semantic Segmentation-based Object Detectionmentioning

confidence: 99%

Characterization of Semantic Segmentation Models on Mobile Platforms for Self-Navigation in Disaster-Struck Zones

Zelek

Jeon

2022

IEEE Access

View full text Add to dashboard Cite

The role of unmanned vehicles for searching and localizing the victims in disaster impacted areas such as earthquake-struck zones is getting more important. Self-navigation on an earthquake zone has a unique challenge of detecting irregularly shaped obstacles such as road cracks, debris on the streets, and water puddles. In this paper, we characterize a number of state-of-the-art Fully Convolutional Network (FCN) models on mobile embedded platforms for self-navigation at these sites containing extremely irregular obstacles. We evaluate the models in terms of accuracy, performance, and energy efficiency. We present a few optimizations for our designed vision system. Lastly, we discuss the trade-offs of these models for a couple of mobile platforms that can each perform self-navigation. To enable vehicles to safely navigate earthquake-struck zones, we compile a new annotated image database of various earthquake impacted regions that is different than traditional road damage databases. We train our database with a number of state-of-the-art semantic segmentation models in order to identify obstacles unique to earthquake-struck zones. Based on the statistics and tradeoffs, an optimal FCN model is selected and applied to the mobile vehicular platforms. To our best knowledge, this is the first study that identifies unique challenges and discusses the accuracy, performance, and energy impact of edge-based self-navigation mobile vehicles for earthquake-struck zones. Our proposed database and trained models are publicly available.INDEX TERMS Convolutional Neural Networks, Edge Computing, Self Driving, Semantic Segmentation I. INTRODUCTIONWith global-wide monitoring and technology evolution, the riskiness of natural disaster is getting lower. However, there are still deadly incidents such as earthquakes. Earthquakes are a geologic inevitability of some countries and states. To reduce the effects from earthquakes, we should think about the ways to quickly search earthquake-struck zones and safely rescue more lives. For this purpose, unmanned autonomous vehicles will be effective, which navigate impacted sites while localizing the people and reporting the damages without risking the lives of others, such as firefighters and other rescue workers. In the natural disaster impacted sites, edge-based small automobiles would be more cost effective and feasible solutions because these small cars can navigate underneath collapsed buildings, unstable bridges, and narrow walkways that are difficult to be driven in by a full-sized vehicle, and also hidden from the viewing angles of aerial solutions such as drones. Though the self-navigation research and industry has been quickly evolving, most of the solutions focus on detecting fairly regular-shaped objects such as hu-mans, buildings, and streets from a full-sized vehicle's perspective. However, these solutions cannot be directly applied to earthquake-struck zones because of the unique obstacles such as debris, cracks, and puddles on the streets, which the mobile edge vehicle perceive...

show abstract

Section: B Semantic Segmentation-based Object Detectionmentioning

confidence: 99%

Characterization of Semantic Segmentation Models on Mobile Platforms for Self-Navigation in Disaster-Struck Zones

Zelek

Jeon

2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Kim et al [1] trained a classifier with hand-crafted color and geometric features using a robot's experience of traversals. We presented an image-based deep neural network for object traversability estimation and its manual annotation-free training method [2]. In [2], we reported some failure cases of navigation due to misclassifications.…”

Section: A Traversability Estimationmentioning

confidence: 99%

“…We presented an image-based deep neural network for object traversability estimation and its manual annotation-free training method [2]. In [2], we reported some failure cases of navigation due to misclassifications. Such machine learning-based methods inevitably suffer from misclassifications.…”

Section: A Traversability Estimationmentioning

confidence: 99%

“…Our previously proposed scene recognition model considering traversable plants [2] consists of two modules; Semantic Segmentation Module (SSM) and Traversability Estimation Module (TEM). SSM provides information of general object classes, namely plant, artificial object, and ground.…”

Section: A Traversability Estimationmentioning

confidence: 99%

“…A pioneering work by Kim et al [1] estimated the traversability of the regions in an image using color and geometric features. The authors' previous work [2] proposed a novel deep neural network (DNN) based on semantic segmentation for object traversability estimation with a manual annotation-free training method. Although we demonstrated the effectiveness of the method through navigation experiments in a plant-rich greenhouse, misclassifications were a critical problem.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Online Refinement of a Scene Recognition Model for Mobile Robots by Observing Human's Interaction with Environments

Matsuzaki¹,

Masuzawa²,

Miura³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

This paper describes a method of online refinement of a scene recognition model for robot navigation considering traversable plants, flexible plant parts which a robot can push aside while moving. In scene recognition systems that consider traversable plants growing out to the paths, misclassification may lead the robot to getting stuck due to the traversable plants recognized as obstacles. Yet, misclassification is inevitable in any estimation methods. In this work, we propose a framework that allows for refining a semantic segmentation model on the fly during the robot's operation. We introduce a fewshot segmentation based on weight imprinting for online model refinement without fine-tuning. Training data are collected via observation of a human's interaction with the plant parts. We propose novel robust weight imprinting to mitigate the effect of noise included in the masks generated by the interaction. The proposed method was evaluated through experiments using real-world data and shown to outperform an ordinary weight imprinting and provide competitive results to fine-tuning with model distillation while requiring less computational cost.The first author is supported in part through the Leading Graduate School Program, "Innovative program for training brain-scienceinformation-architects by analysis of massive quantities of highly technical information about the brain," by the Ministry of Education, Culture, Sports,

show abstract

Overview of Terrain Traversability Evaluation for Autonomous Robots

Shu,

Dong,

Liu

et al. 2024

Journal of Field Robotics

View full text Add to dashboard Cite

Traversability evaluation is the foundation and core of unmanned platforms for scene understanding and autonomous navigation, whose successful completion relies on the analysis of the platform's characteristics and the semantic and geometric features of the surrounding environment. This topic has been reviewed by many literatures, which are characterized by a single perspective and lack comprehensive evaluation frameworks. Thus, the concept and developmental trajectory of traversability evaluation are initially outlined in this paper, distinguishing it from other issues, while constructing an evaluation framework based on two categories: direct assessment and downstream task assessment. Subsequently, traversability evaluation methods are classified based on multiple dimensions, including sensor types, robot types, usage scenarios, and learning approaches. On the basis of the constructed evaluation framework, comparisons are made among existing algorithms in terms of performance and runtime. Subsequently, a summary is provided on commonly used features and their mainstream computation methods in terrain evaluation. Additionally, open‐source data sets in this field and projects for scene construction and algorithm validation are compiled and organized. Finally, an analysis is conducted on the development direction and trends, emphasizing the urgent need to establish standardized evaluation systems and comparison baselines. Furthermore, it is imperative that various environmental and platform information be comprehensively integrated into algorithms, while also ensuring that simulation, demonstration, and exploration are incorporated into a unified framework to enhance the robot's learning capability.

show abstract

Image-Based Scene Recognition for Robot Navigation Considering Traversable Plants and Its Manual Annotation-Free Training

Cited by 14 publications

References 49 publications

Characterization of Semantic Segmentation Models on Mobile Platforms for Self-Navigation in Disaster-Struck Zones

Characterization of Semantic Segmentation Models on Mobile Platforms for Self-Navigation in Disaster-Struck Zones

Online Refinement of a Scene Recognition Model for Mobile Robots by Observing Human's Interaction with Environments

Overview of Terrain Traversability Evaluation for Autonomous Robots

Contact Info

Product

Resources

About