Transforming multiple visual surveys of a natural environment into time-lapses

Abstract

This article presents a new framework to help transform visual surveys of a natural environment into time-lapses. As data association across year-long variation in appearance continues to represent a formidable challenge, we present success with a map-centric approach, which builds on 3D vision for visual data association. We use a foundation of map point priors and geometric constraints within a dense correspondence image alignment optimization to align images and acquire loop closures between surveys. This framework produces many loop closures between sessions. Outlier loop closures are filtered in the frontend and in the backend to improve robustness. From the result map, the Reprojection Flow algorithm is applied to create time-lapses. The evaluation of our framework on the Symphony Lake Dataset, which has considerable variation in appearance, led to year-long time-lapses of many different scenes. In comparison with another approach based on using iterative closest point (ICP) plus a homography, our framework produced more and better-quality alignments. With many scenes of the 1.3 km environment consistently aligning well in random image pairs, we next produced 100 time-lapses across 37 surveys captured in a year. Approximately one-third had at least 20 (out of usually 33) well-aligned images, which spanned all four seasons. With promising results, we evaluated the pose error of misaligned image pairs and found that improving map consistency could lead to even better results.

Bridge inspection component registration for damage evolution

“…This dataset of land surveys is called the Symphony Lake Dataset. 21 Griffith et al 20 used camera position information, SIFT keypoints, and an optimization process to warp images to fit to the target image. The result was a time-lapse of specific locations along the route for each target location.…”

“…The result was a time-lapse of specific locations along the route for each target location. Griffith 20 recommended to include a more robust and modern method for collecting keypoints in the future. The research presented in this paper uses a more modern keypoint finder, SuperGlue, 6 with a similar goal as Griffith et al, except that instead of creating time-lapse imagery, we aim to detect and document damage progression over time for bridge inspectors.…”

“…To the authors' knowledge, the research that is most closely related to the work described here involved aligning images taken from a robot, GeorgiaTech-Loraine (GTL) ClearpathKingfisher, on a lake for a land survey repeated over 130 times over the span of 3.5 years 20 (see Figure 3 of Ref. 20). The robot navigated approximately the same path each time it recorded data throughout the year.…”