Pose Consistency KKT-Loss for Weakly Supervised Learning of Robot-Terrain Interaction Model

Abstract

We address the problem of self-supervised learning for predicting the shape of supporting terrain (i.e. the terrain which will provide rigid support for the robot during its traversal) from sparse input measurements. The learning method exploits two types of ground-truth labels: dense 2.5D maps and robot poses, both estimated by a usual SLAM procedure from offline recorded measurements. We show that robot poses are required because straightforward supervised learning from the 3D maps only suffers from: (i) exaggerated height of the supporting terrain caused by terrain flexibility (vegetation, shallow water, snow or sand) and (ii) missing or noisy measurements caused by high spectral absorbance or non-Lambertian reflectance of the measured surface. We address the learning from robot poses by introducing a novel KKT-loss, which emerges as the distance from necessary Karush-Kuhn-Tucker conditions for constrained local optima of a simplified first-principle model of the robot-terrain interaction. We experimentally verify that the proposed weakly supervised learning from ground-truth robot poses boosts the accuracy of predicted support heightmaps and increases the accuracy of estimated robot poses. All experiments are conducted on a dataset captured by a real platform. Both the dataset and codes which replicates experiments in the paper are made publicly available as a part of the submission.

Spatial Group-Wise Enhance: Enhancing Semantic Feature Learning in CNN

“…Rather than using heuristic modeling, Šalanský et al [17] introduced a learning-based method to filter a 2.5D map, while Wellington et al [18] employ a learning approach to filter a voxel map. In our research, instead of using a fused representation such as an elevation map or voxel map, we refine the raw sensor measurements directly.…”

TerrainNet: Visual Modeling of Complex Terrain for High-speed, Off-road Navigation

“…In practice, obstacles or terrain discontinuities leave areas with missing values in the elevation map, leading to suboptimal motion planning. Inspired by data-driven image in-painting methods, recent works [36,41] propose self-supervised learning to reconstruct the occluded area in an input elevation map.…”

Geometry‐based flipper motion planning for articulated tracked robots traversing rough terrain in real‐time

“…However, they are mainly used for passability analysis in global path planning because they cannot arbitrarily specify the flipper angle when predicting the robot pose, and the robot center will shift during iteration. Recently, Šalanský et al (2021) proposed a weakly supervised learning method for pose prediction, which trains a robot–terrain interaction model with the ground truth of robot position and pose. A KKT‐Loss is designed for the neural network to predict the robot pose with the minimum potential energy.…”

“…On the basis of the predicted robot poses, the influence of flippers can be then evaluated, which will greatly support the automatic flipper motion planning algorithm. Multiple pose prediction methods have been proposed, such as physical simulation methods (Jun et al, 2015; Norouzi et al, 2017), iterative geometry methods (Brunner et al, 2015; Fabian et al, 2020), and machine learning methods (Šalanský, 2021; Šalanský et al, 2021). However, existing methods are either too computationally intensive, sensitive to the initial values, or unable to predict the robot pose at specified flipper angles.…”