TerrainNet: Visual Modeling of Complex Terrain for High-speed, Off-road Navigation

Abstract

Effective use of camera-based vision systems is essential for robust performance in autonomous off-road driving, particularly in the high-speed regime. Despite success in structured, on-road settings, current end-to-end approaches for scene prediction have yet to be successfully adapted for complex outdoor terrain. To this end, we present TerrainNet, a visionbased terrain perception system for semantic and geometric terrain prediction for aggressive, off-road navigation. The approach relies on several key insights and practical considerations for achieving reliable terrain modeling. The network includes a multi-headed output representation to capture fine-and coarsegrained terrain features necessary for estimating traversability. Accurate depth estimation is achieved using self-supervised depth completion with multi-view RGB and stereo inputs. Requirements for real-time performance and fast inference speeds are met using efficient, learned image feature projections. Furthermore, the model is trained on a large-scale, real-world off-road dataset collected across a variety of diverse outdoor environments. We show how TerrainNet can also be used for costmap prediction and provide a detailed framework for integration into a planning module. We demonstrate the performance of TerrainNet through extensive comparison to current state-of-the-art baselines for camera-only scene prediction. Finally, we showcase the effectiveness of integrating TerrainNet within a complete autonomousdriving stack by conducting a real-world vehicle test in a challenging off-road scenario.

ForestTrav: 3D LiDAR-Only Forest Traversability Estimation for Autonomous Ground Vehicles

“…Semantic and spatial cost maps (e.g. semantics, height difference, ceiling height) for high-speed off-road driving using stereo-vision has been successfully implemented [17]. The authors emphasize fast inference time at the cost of (geometric) representation fidelity and aim to avoid complex areas.…”

“…To classify an environment as vegetated is inherently subjective. In different works it can encompass natural areas devoid of overhanging foliage, a large area with a few trees [17] or an area with a cluster of trees in a park or forest void of undergrowth [9]. Consequently, there is a large variation in works presented in the context of vegetated environments, or more specifically forests.…”