An information gain formulation for active volumetric 3D reconstruction

Isler, Stefan; Sabzevari, Reza; Delmerico, Jeffrey A.; Scaramuzza, Davide

doi:10.1109/icra.2016.7487527

Cited by 134 publications

(148 citation statements)

References 17 publications

Supporting

Mentioning

146

Contrasting

Order By: Relevance

“…With respect to the information metric, Isler et al 25 propose several information gain-based metrics; in this article, we have implemented one of their metrics and have compared both information metrics. Isler et al 25 mention that the robot interface, in charge of the robot motion and inverse kinematics, should provide the promising state candidates; however, there are no details on how it is done.…”

Section: Related Workmentioning

confidence: 99%

Tree-based search of the next best view/state for three-dimensional object reconstruction

Vasquez-Gomez

Sucar

Murrieta-Cid

et al. 2018

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

Three-dimensional models from real objects have many applications in robotics. To automatically build a three-dimensional model from an object, it is essential to determine where to place the range sensor in order to completely observe the object. However, the view (position and orientation) of the sensor is not sufficient, given that its corresponding robot state needs to be calculated. Additionally, a collision-free trajectory to reach that state is required. In this article, we directly find the state of the robot whose corresponding sensor view observes the object. This method does not require to calculate the inverse kinematics of the robot. Unlike previous approaches, the proposed method guides the search with a tree structure based on a rapidly exploring random tree overcoming previous sampling techniques. In addition, we propose an information metric that improves the reconstruction performance of previous information metrics.

show abstract

Section: Related Workmentioning

confidence: 99%

Tree-based search of the next best view/state for three-dimensional object reconstruction

Vasquez-Gomez

Sucar

Murrieta-Cid

et al. 2018

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

show abstract

“…We define VI as the amount of information a single voxel is expected to provide when seen from a particular view Isler et al (2016). The next best view (NBV) is the view that maximizes this metric, minus any estimated costs.…”

Section: Volumetric Informationmentioning

confidence: 99%

“…Conceptually, our systems build upon the framework presented in Isler et al (2016): it consists of several independent modules that interact through well-defined interfaces, yielding a very flexible system architecture, as shown in Fig. 2.…”

Section: System Overviewmentioning

confidence: 99%

“…This may be a static set that is fixed a priori, or a dynamic set that is recreated and evaluated at each iteration. Static viewspaces are usually sampled from simple geometries circumscribed around the object, such as a cylinder Pito (1999), a sphere Trummer et al (2010), or a combination of cylinder and hemisphere Isler et al (2016). Dynamic viewspaces are generated based on the current map and the possible poses of the robot Kriegel et al (2015), Wettach and Berns (2010), or may also be sampled from a fixed geometry, but randomly resampled with every iteration .…”

Section: System Overviewmentioning

confidence: 99%

“…completeness, entropy), which focus the reconstruction on observing unexplored regions or refining already observed ones. The reconstruction approach and software framework described in this paper was originally proposed in Isler et al (2016), but the experimental evaluation here is significantly expanded.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A comparison of volumetric information gain metrics for active 3D object reconstruction

et al. 2017

Self Cite

View full text Add to dashboard Cite

In this paper, we investigate the following question: when performing next best view selection for volumetric 3D reconstruction of an object by a mobile robot equipped with a dense (camera-based) depth sensor, what formulation of information gain is best? To address this question, we propose several new ways to quantify the volumetric information (VI) contained in the voxels of a probabilistic volumetric map, and compare them to the state of the art with extensive simulated experiments. Our proposed formulations incorporate factors such as visibility likelihood and the likelihood of seeing new parts of the object. The results of our experiments allow us to draw some clear conclusions about the VI formulations that are most effective in different mobile-robot reconstruction scenarios. To the best of our knowledge, this is the first comparative survey of VI formulation performance for active 3D object reconstruction. Additionally, our modular software framework is adaptable to other robotic platforms and general reconstruction problems, and we release it open source for autonomous reconstruction tasks. Abstract In this paper, we investigate the following question: when performing next best view selection for volumetric 3D reconstruction of an object by a mobile robot equipped with a dense (camera-based) depth sensor, what formulation of information gain is best? To address this question, we propose several new ways to quantify the volumetric information (VI) contained in the voxels of a probabilistic volumetric map, and compare them to the state of the art with extensive simulated experiments. Our proposed formulations incorporate factors such as visibility likelihood and the likelihood of seeing new parts of the object. The results of our experiments allow us to draw some clear conclusions about the VI formulations that are most effective in different mobile-robot reconstruction scenarios. To the best of our knowledge, this is the first comparative survey of VI formulation performance for active 3D object reconstruction. Additionally, our modular software framework is adaptable to other robotic platforms and general reconstruction problems, and we release it open source for autonomous reconstruction tasks.

show abstract

Autonomous aerial cinematography in unstructured environments with learned artistic decision‐making

Bonatti

Wang

et al. 2020

Journal of Field Robotics

View full text Add to dashboard Cite

Aerial cinematography is revolutionizing industries that require live and dynamic camera viewpoints such as entertainment, sports, and security. However, safely piloting a drone while filming a moving target in the presence of obstacles is immensely taxing, often requiring multiple expert human operators. Hence, there is a demand for an autonomous cinematographer that can reason about both geometry and scene context in real-time. Existing approaches do not address all aspects of this problem; they either require high-precision motion-capture systems or global positioning system tags to localize targets, rely on prior maps of the environment, plan for short time horizons, or only follow fixed artistic guidelines specified before the flight. In this study, we address the problem in its entirety and propose a complete system for real-time aerial cinematography that for the first time combines: (a) vision-based target estimation; (b) 3D signed-distance mapping for occlusion estimation; (c) efficient trajectory optimization for long time-horizon camera motion; and (d) learning-based artistic shot selection.We extensively evaluate our system both in simulation and in field experiments by filming dynamic targets moving through unstructured environments. Our results indicate that our system can operate reliably in the real world without restrictive assumptions. We also provide in-depth analysis and discussions for each module, with the hope that our design tradeoffs can generalize to other related applications. Videos of the complete system can be found at https://youtu.be/ookhHnqmlaU. K E Y W O R D S aerial robotics, cinematography, computer vision, learning, mapping, motion planning Within the filming context, this cost function measures jerkiness of motion, safety, environmental occlusion of the actor, and shot quality (artistic quality of viewpoints). This cost function depends on the environment and , and on the actor forecast ξ a , all of which are sensed on-the-fly. The changing nature of the environment and ξ a demands replanning at a high frequency. Here we briefly touch upon the four components of the cost function J(ξ q ) (refer to Section 7 for details and mathematical expressions): Smoothness J smooth (ξ q ): Penalizes jerky motions that may lead to camera blur and unstable flight; Safety J obs (ξ q , ): Penalizes proximity to obstacles that are unsafe for the UAV; Occlusion J occ (ξ q , ξ a , ): Penalizes occlusion of the actor by obstacles in the environment; Shot quality J shot (ξ q , ξ a , Ω art ): Penalizes poor viewpoint angles and scales that deviate from the desired artistic guidelines, given by the set of parameters Ω art . In its simplest form, we can express J(ξ q ) as a linear composition of each individual cost, weighted by scalars λ i . The objective is to * = ( ) ()={ } J J J J J J xyz J J J J J J x y z J occ + J obs 99.4 ± 2.2 94.2 ± 7.3 86.9 ± 9.3 J obs 98.8 ± 3.0 87.1 ± 8.5 75.3 ± 11.8 Avg. dist. to ξ shot (m) J occ + J obs 0.4 ± 0.4 6.2 ± 11.2 10.7 ± 13.2 J obs 0.05 ± 0.1 0.3 ± 0.2 0.5 ± 0...

show abstract

An information gain formulation for active volumetric 3D reconstruction

Cited by 134 publications

References 17 publications

Tree-based search of the next best view/state for three-dimensional object reconstruction

Tree-based search of the next best view/state for three-dimensional object reconstruction

A comparison of volumetric information gain metrics for active 3D object reconstruction

Autonomous aerial cinematography in unstructured environments with learned artistic decision‐making

Contact Info

Product

Resources

About