Terrain Segmentation and Roughness Estimation using RGB Data: Path Planning Application on the CENTAURO Robot

Abstract

Robots operating in real world environments require a high-level perceptual understanding of the chief physical properties of the terrain they are traversing. In unknown environments, roughness is one such important terrain property that could play a key role in devising robot control/planning strategies. In this paper, we present a fast method for predicting pixel-wise labels of terrain (stone, sand, road/sidewalk, wood, grass, metal) and roughness estimation, using a single RGBbased deep neural network. Real world RGB images are used to experimentally validate the presented approach. Furthermore, we demonstrate an application of our proposed method on the centaur-like wheeled-legged robot CENTAURO, by integrating it with a navigation planner that is capable of re-configuring the leg joints to modify the robot footprint polygon for stability purposes or for safe traversal among obstacles.

Pose Consistency KKT-Loss for Weakly Supervised Learning of Robot-Terrain Interaction Model

“…All compared methods use the same SLAM procedure [11] to align the input measurements and the pose is always predicted from estimated supporting terrain ĥ by pose predicting network g ω . g ω is trained 5 to minimize L 2 (φ, g ω ( ĥ)). A detailed description of implemented methods is in the following paragraphs.…”

“…Self-supervised learning: Other methods learn to predict the robot-terrain interaction directly. Methods such as Suryamurthy et al [5] learn to estimate terrain roughness from RGB images for wheeled Centauro robot, where terrain roughness labels are automatically computed from SfM optimized heightmaps. In contrast to us, such an approach inherently suffers from the inability to assess the terrain's flexibility and the inability to reconstruct visually homogeneous terrain.…”

Learning-Based Methods of Perception and Navigation for Ground Vehicles in Unstructured Environments: A Review

“…Suryamurthy et al [ 65 ] proposes a fast method for terrain semantic recognition-segmentation and roughness estimation from RGB images. The authors present a network architecture composed of a primary CNN for terrain classification and a secondary module (based on RGB features) for roughness regression.…”

Agile Legged-Wheeled Reconfigurable Navigation Planner Applied on the CENTAURO Robot

“…It was possible to force the robot to both change its polygon only in unavoidable situations as well as to prefer polygon changes if it lead to a shorter traversal path. This work was further combined with a roughness estimation algorithm in [20] to allow for motion planning over low-lying rough surfaces.…”