Semantic-aware Spatiotemporal Alignment of Natural Outdoor Surveys

Abstract

This article presents a keyframe-based, innovative map registration scheme for applications that benefit from recurring data acquisition, such as long-term natural environment monitoring. The proposed method consists of a multistage pipeline, in which semantic knowledge of the scene is acquired using a pretrained neural network. The semantic knowledge is subsequently employed to constrain the Iterative Closest Point algorithm (ICP). In this article, semantic-aware ICP is used to build keyframes as well as to align them both spatially and temporally, with neighboring keyframes and those captured around the same area but at a different point in time, respectively. Hierarchical clustering of ICP-generated transformations is then used to both eliminate outliers and find alignment consensus, followed by an optimization scheme based on a factor graph that includes loop closure. To evaluate the proposed framework, data were captured using a portable robotic sensor suite consisting of three cameras, a three-dimensional lidar, and an inertial navigation system. The data were acquired monthly over 12 months by revisiting the same trajectory between August 2020 and July 2021.

Comparing metrics for evaluating 3D map quality in natural environments

“…In robotics, the maps are commonly built with the primary objective of optimal navigation planning, such as [7]. However, there are cases where the purpose is to reconstruct a scene to enable its monitoring, such as [8]. When the purpose of mapping is to reconstruct a scene, the most common maps are volumetric maps, or 3D-grids, and meshes, or 3D-surface maps.…”

“…Among the most widely used techniques, we can cite Ball-Pivoting Algorithm [9], Poisson Surface Reconstruction [10] or Delaunay triangulations [11], mainly applied to mesh a single object. Other methods tackle large-scale meshing, such as [7] who includes texture in the mesh to improve its visual aspect, or [8] who build a semantic mesh representation of a large-scale environment with the Las Vegas Reconstruction Toolkit [12]. Recently, other methods have been presented to build meshes with deep learning techniques, such as Voxel2Mesh [13].…”