Locomotion Policy Guided Traversability Learning using Volumetric Representations of Complex Environments

Abstract

Autonomous navigation at high speeds in off-road environments necessitates robots to comprehensively understand their surroundings using onboard sensing only. The extreme conditions posed by the off-road setting can cause degraded camera image quality due to poor lighting and motion blur, as well as limited sparse geometric information available from LiDAR sensing when driving at high speeds. In this work, we present RoadRunner, a novel framework capable of predicting terrain traversability and an elevation map directly from camera and LiDAR sensor inputs. RoadRunner enables reliable autonomous navigation, by fusing sensory information, handling of uncertainty, and generation of contextually informed predictions about the geometry and traversability of the terrain while operating at low latency. In contrast to existing methods relying on classifying handcrafted semantic classes and using heuristics to predict traversability costs, our method is trained end-to-end in a self-supervised fashion. The RoadRunner network architecture builds upon popular sensor fusion network architectures from the autonomous driving domain, which embed LiDAR and camera information into a common Bird's Eye View perspective. Training is enabled by utilizing an existing traversability estimation stack to generate training data in hindsight in a scalable manner from realworld off-road driving datasets. Furthermore, RoadRunner improves the system latency by a factor of ∼ 4, from 500 ms to 140 ms, while improving the accuracy for traversability costs and elevation map predictions. We demonstrate the effectiveness of RoadRunner in enabling safe and reliable offroad navigation at high speeds in multiple real-world driving scenarios through unstructured desert environments.

Learning robust autonomous navigation and locomotion for wheeled-legged robots

“…5D). Specifically, the robot was able to traverse higher steps when descending, indicating that it has a more advanced understanding of the terrain compared with cost-map approaches for traversability estimation (21,26). Traditional methods often use symmetric traversability maps that are independent of motion direction, whereas our approach makes decisions on the basis of the current terrain, the robot's state, and the characteristics of the LLC.…”

“…This understanding is crucial for issuing commands that optimize efficiency on flat terrain while maintaining agility when faced with obstacles. Many existing approaches (20,21) are based on explicit navigation costs, such as traversability (21,22), without considering the robot's whole-body states. They focus on generating kinematic navigation plans by sampling-based planning on these estimated cost maps.…”

Uncertainty-aware visually-attentive navigation using deep neural networks

“…A body of work utilizes deep learning to derive interpretable maps, which are then used by classical planners to plan collision-free paths Wang et al (2021); Frey et al (2022); Castro et al (2023); Zeng et al (2019). Instead of learning classical map representations from raw observation data, many works present methods to encode raw sensor data into an implicit latent vector Hoeller et al (2021); Dugas et al (2021); Ichter and Pavone (2019); Srinivas et al (2018); Qureshi et al (2021).…”

ForestTrav: 3D LiDAR-Only Forest Traversability Estimation for Autonomous Ground Vehicles

“…The performance of the method also showed limitations in the highly challenging densely vegetated environments such as those addressed in this work. sparse convolutional neural networks (SCNNs) have shown low inference times for sparse voxel representations, which are suitable for TE estimation and scene completion in structured environments [13]. However, their use has not been explored for TE in vegetated environments.…”