Automated view and path planning for scalable multi-object 3D scanning

Fan, Xiangyu; Zhang, Linguang; Brown, Benedict J.; Rusinkiewicz, Szymon

doi:10.1145/2980179.2980225

Cited by 33 publications

(16 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our system highlights the boundaries of the reconstructed model after each scan is integrated (Figure 12, left), suggesting where data is required and helping to plan the next scan. While more advanced next-best-view optimizations could be integrated [Fan et al 2016;Wu et al 2014], we found the guidance provided by our direct visual feedback to be sufficient for all our experiments. We efficiently scanned models with complex shapes, combining an initial set of automatic scans (taken using a rotational stage), with a few manual scans of the occluded regions (Figure 12, right).…”

Section: Resultsmentioning

confidence: 97%

Field-aligned online surface reconstruction

et al. 2017

View full text Add to dashboard Cite

Today's 3D scanning pipelines can be classified into two overarching categories: ofine, high accuracy methods that rely on global optimization to reconstruct complex scenes with hundreds of millions of samples, and online methods that produce real-time but low-quality output, usually from structure-from-motion or depth sensors. The method proposed in this paper is the first to combine the benefits of both approaches, supporting online reconstruction of scenes with hundreds of millions of samples from high-resolution sensing modalities such as structured light or laser scanners. The key property of our algorithm is that it sidesteps the signed-distance computation of classical reconstruction techniques in favor of direct filtering, parametrization, and mesh and texture extraction. All of these steps can be realized using only weak notions of spatial neighborhoods, which allows for an implementation that scales approximately linearly with the size of each dataset that is integrated into a partial reconstruction. Combined, these algorithmic differences enable a drastically more efficient output-driven interactive scanning and reconstruction workflow, where the user is able to see the final quality field-aligned textured mesh during the entirety of the scanning procedure. Holes or parts with registration problems are displayed in real-time to the user and can be easily resolved by adding further localized scans, or by adjusting the input point cloud using our interactive editing tools with immediate visual feedback on the output mesh. We demonstrate the effectiveness of our algorithm in conjunction with a state-of-the-art structured light scanner and optical tracking system and test it on a large variety of challenging models

show abstract

Section: Resultsmentioning

confidence: 97%

Field-aligned online surface reconstruction

et al. 2017

View full text Add to dashboard Cite

show abstract

“…Some works (e.g. (Fan et al, 2016;Ahmadabadian et al, 2014a,b;Trummer et al, 2010)) thus focus on the reconstruction of small scale scenes in laboratories. For this kind of approaches we refer the reader to a recent review article of Karaszewski et al (2016).…”

Section: Related Workmentioning

confidence: 99%

Prioritized multi-view stereo depth map generation using confidence prediction

Mostegel

Fraundorfer

Bischof

2018

ISPRS Journal of Photogrammetry and Remote Sensing

View full text Add to dashboard Cite

In this work, we propose a novel approach to prioritize the depth map computation of multi-view stereo (MVS) to obtain compact 3D point clouds of high quality and completeness at low computational cost. Our prioritization approach operates before the MVS algorithm is executed and consists of two steps. In the first step, we aim to find a good set of matching partners for each view. In the second step, we rank the resulting view clusters (i.e. key views with matching partners) according to their impact on the fulfillment of desired quality parameters such as completeness, ground resolution and accuracy. Additional to geometric analysis, we use a novel machine learning technique for training a confidence predictor. The purpose of this confidence predictor is to estimate the chances of a successful depth reconstruction for each pixel in each image for one specific MVS algorithm based on the RGB images and the image constellation. The underlying machine learning technique does not require any ground truth or manually labeled data for training, but instead adapts ideas from depth map fusion for providing a supervision signal. The trained confidence predictor allows us to evaluate the quality of image constellations and their potential impact to the resulting 3D reconstruction and thus builds a solid foundation for our prioritization approach. In our experiments, we are thus able to reach more than 70% of the maximal reachable quality fulfillment using only 5% of the available images as key views. For evaluating our approach within and across different domains, we use two completely different scenarios, i.e. cultural heritage preservation and reconstruction of single family houses.

show abstract

“…Reinforcement learning has been utilized to estimate camera trajectories for scene exploration [Kollar and Roy 2008), but not for quality 3D reconstruction. For scanning multiple objects, Fan et al [2016] select the best views for each object and then optimize the entire scanning trajectory by solving a Traveling Salesman Problem, with a special scanning setup. lt is, however, unclear how this method could be extended to a mobile robot setting which requires collision avoidance.…”

Section: Related Workmentioning

confidence: 99%

Autonomous reconstruction of unknown indoor scenes guided by time-varying tensor fields

Zheng

Yan

et al. 2017

ACM Trans. Graph.

View full text Add to dashboard Cite

(a) (b) (c) (d) Fig. 1. We introduce an algorithm for the autonomous reconstruction of indoor scenes, based on time-varying tensor fields (a). Given the partially scanned scene, a 2D tensor field is computed on-the-fly over the floor plane, constrained by the partial reconstruction (b). The robot is guided by the field with smooth paths, which are locally formed with field advection (red curve in (a)) and globally planned with the help of the field topology (see the curve networks in (b) and (c), with reconstruction uncertainty color-coded along the curves). The topological structure is well-defined for incomplete scenes (b), suited for guiding exploratory reconstruction of unknown scenes. The output is a full 3D reconstruction (d), at which the topology of the field reflects the scene layout (c).Autonomous reconstruction of unknown scenes by a mobile robot inherently poses the question of balancing between exploration efficacy and reconstruction quality. We present a navigation-by-reconstruction approach to address this question, where moving paths of the robot are planned to account for both global efficiency for fast exploration and local smoothness to obtain high-quality scans. An RGB-D camera, attached to the robot arm, is dictated by the desired reconstruction quality as well as the movement of the robot itself. Our key idea is to harness a time-varying tensor field to guide robot movement, and then solve for 3D camera control under the constraint of the 2D robot moving path. The tensor field is updated in real time, conforming to the progressively reconstructed scene. We show that tensor fields are well suited for guiding autonomous scanning for two reasons: first, they contain sparse and controllable singularities that allow generating a locally smooth robot path, and second, their topological structure can be used for globally efficient path routing within a partially reconstructed scene. We have conducted numerous tests with a mobile robot, and demonstrate that our method leads to a smooth exploration and high-quality reconstruction of unknown indoor scenes.

show abstract

Automated view and path planning for scalable multi-object 3D scanning

Cited by 33 publications

References 23 publications

Field-aligned online surface reconstruction

Field-aligned online surface reconstruction

Prioritized multi-view stereo depth map generation using confidence prediction

Autonomous reconstruction of unknown indoor scenes guided by time-varying tensor fields

Contact Info

Product

Resources

About