Large-scale outdoor scene reconstruction and correction with vision

Abstract

We provide the theory and the system needed to create large-scale dense reconstructions for mobile-robotics applications: this stands in contrast to the object-centric reconstructions dominant in the literature. Our BOR2G system fuses data from multiple sensor modalities (cameras, lidars, or both) and regularizes the resulting 3D model. We use a compressed 3D data structure, which allows us to operate over a large scale. In addition, because of the paucity of surface observations by the camera and lidar sensors, we regularize over both two (camera depth maps) and three dimensions (voxel grid) to provide a local contextual prior for the reconstruction. Our regularizer reduces the median error between 27% and 36% in 7.3 km of dense reconstructions with a median accuracy between 4 and 8 cm. Our pipeline does not end with regularization. We take the unusual step to apply a learned correction mechanism that takes the global context of the reconstruction and adjusts the constructed mesh, addressing errors that are pathological to the first-pass camera-derived reconstruction. We evaluate our system using the Stanford Burghers of Calais, Imperial College ICL-NUIM, Oxford Broad Street (released with this paper), and the KITTI datasets. These latter datasets see us operating at a combined scale and accuracy not seen in the literature. We provide statistics for the metric errors in all surfaces created compared with those measured with 3D lidar as ground truth. We demonstrate our system in practice by reconstructing the inside of the EUROfusion Joint European Torus (JET) fusion reactor, located at the Culham Centre for Fusion Energy (UK Atomic Energy Authority) in Oxfordshire.

The UMA-SAR Dataset: Multimodal data collection from a ground vehicle during outdoor disaster response training exercises

“…Liu et al (2018) and Dubé et al (2020) addressed 3D lidar-based SLAM methods on natural terrains and semi-unstructured environments, respectively, but not in collapsed, cluttered, and unstructured outdoor environments where assumptions such as segments and planes are not valid. Recent works on largescale and real-time visual SLAM show high-quality maps and loop-closing in outdoor urban environments (Lynen et al, 2020;Tanner et al, 2020), where GPS and 3D lidar are used as ground truth. Moreover, works in urban environments using vision, semantic mapping (Cadena et al, 2016), and including TIR images (Shin and Kim, 2019) offer promising techniques for potential SLAM applications in disaster robotics.…”

Elastic and Efficient LiDAR Reconstruction for Large-Scale Exploration Tasks

“…For instance, Niessner et al [16] employed a TSDF for large-scale mapping by exploiting the sparsity of environment via a technique called Hashing Voxel Grid (HVG). Tanner et al [17] also utilised HVG for efficient large-scale TSDF reconstruction over kilometers. Their pipeline BOR 2 G further incorporates multiple types of sensor inputs, including long-range LiDAR.…”