TNS: Terrain Traversability Mapping and Navigation System for Autonomous Excavators

Abstract

We present Terrain Traversability Mapping (TTM), a real-time mapping approach for terrain traversability estimation and path planning for autonomous excavators in an unstructured environment. We propose an efficient learningbased geometric method to extract terrain features from RGB images and 3D pointclouds and incorporate them into a global map for planning and navigation for autonomous excavation. Our method used the physical characteristics of the excavator, including maximum climbing degree and other machine specifications, to determine the traversable area. Our method can adapt to changing environments and update the terrain information in real-time. Moreover, we prepare a novel dataset, Autonomous Excavator Terrain (AET) dataset, consisting of RGB images from construction sites with seven categories according to navigability. We integrate our mapping approach with planning and control modules in an autonomous excavator navigation system, which outperforms previous method by 49.3% in terms of success rate based on existing planning schemes. With our mapping the excavator can navigate through unstructured environments consisting of deep pits, steep hills, rock piles, and other complex terrain features.

TerraPN: Unstructured Terrain Navigation using Online Self-Supervised Learning

“…To this end, there have been several works in pixelwise semantic segmentation to classify a terrain into multiple predefined traversability classes (smooth, rough, bumpy, forbidden, etc.) [3], [38], [39]. These works are typically trained in a supervised manner using human-annotated image datasets.…”

TerraPN: Unstructured Terrain Navigation using Online Self-Supervised Learning

“…1) Supervised Methods: Works in pixel-wise semantic segmentation classify a terrain into multiple predefined classes such as traversable, non-traversable, obstacle, forbidden, etc. [1], [2], [3]. Fusing a terrain's semantic (visual) and geometric (point cloud) features for better classification has also been studied [3].…”

“…Therefore, a precursor to smooth robot navigation on different terrains is perceiving and learning their surface properties. To this end, several works in computer vision, specifically semantic segmentation [1], [2], [3], have demonstrated good terrain classification capabilities on RGB images. However, they rely on large hand-labeled datasets, which do not account for a robot's properties and might misclassify a traversable terrain for a robot as non-traversable.…”

Lunar ground segmentation using a modified U-net neural network

“…In [16], authors propose a lightweight neural network for terrain semantic segmentation focused on unstructured environments which is capable of merging multi-scale visual features, in order to efficiently group and classify different types of terrains while a reinforcement learning algorithm, is able to utilize the predicted segmentation maps aiming to plan and guide a robot in paths with high safety. Similarly, in [17], a real-time terrain mapping method for autonomous excavators is presented, which is able to provide semantic and geometric information for the terrain using RGB images and 3D point cloud data, while a dataset which includes images from construction sites is designed and utilized. Regarding the datasets for earthy unstructured environments, in [18,19], two publicly available datasets were developed for semantic segmentation deep learning models, focusing on self-driving in semiunstructured or dense-vegetated environments.…”