Learning to Reconstruct 3D Structures for Occupancy Mapping

Guizilini, Vitor; Ramos, Fábio

doi:10.15607/rss.2017.xiii.009

Cited by 17 publications

(11 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Brock et al (2016) also present successful results using variational autoencoders for reconstructing voxelized 3D data. Different configurations of encoding and decoding networks have also been proposed for achieving localization and for reconstructing and completing 3D shapes and environments (Dai et al, 2017; Elbaz et al, 2017; Guizilini and Ramos, 2017; Ricao Canelhas et al, 2017; Schönberger et al, 2018; Varley et al, 2017).…”

Section: Related Workmentioning

confidence: 99%

SegMap: Segment-based mapping and localization using data-driven descriptors

Dubé

Cramariuc

Dugas

et al. 2019

The International Journal of Robotics Research

173

View full text Add to dashboard Cite

Precisely estimating a robot’s pose in a prior, global map is a fundamental capability for mobile robotics, e.g., autonomous driving or exploration in disaster zones. This task, however, remains challenging in unstructured, dynamic environments, where local features are not discriminative enough and global scene descriptors only provide coarse information. We therefore present SegMap: a map representation solution for localization and mapping based on the extraction of segments in 3D point clouds. Working at the level of segments offers increased invariance to view-point and local structural changes, and facilitates real-time processing of large-scale 3D data. SegMap exploits a single compact data-driven descriptor for performing multiple tasks: global localization, 3D dense map reconstruction, and semantic information extraction. The performance of SegMap is evaluated in multiple urban driving and search and rescue experiments. We show that the learned SegMap descriptor has superior segment retrieval capabilities, compared with state-of-the-art handcrafted descriptors. As a consequence, we achieve a higher localization accuracy and a 6% increase in recall over state-of-the-art handcrafted descriptors. These segment-based localizations allow us to reduce the open-loop odometry drift by up to 50%. SegMap is open-source available along with easy to run demonstrations.

show abstract

Section: Related Workmentioning

confidence: 99%

SegMap: Segment-based mapping and localization using data-driven descriptors

Dubé

Cramariuc

Dugas

et al. 2019

The International Journal of Robotics Research

173

View full text Add to dashboard Cite

show abstract

“…Brock et al [2] also present successful results using variational autoencoders for reconstructing voxelized 3D data. Different configurations of encoding and decoding networks have also been proposed for achieving localization and for reconstructing and completing 3D shapes and environments [6,10,15,26,28,31].…”

Section: Related Workmentioning

confidence: 99%

SegMap: 3D Segment Mapping using Data-Driven Descriptors

Dubé¹,

Cramariuc²,

Dugas³

et al. 2018

Robotics: Science and Systems XIV

141

View full text Add to dashboard Cite

When performing localization and mapping, working at the level of structure can be advantageous in terms of robustness to environmental changes and differences in illumination. This paper presents SegMap: a map representation solution to the localization and mapping problem based on the extraction of segments in 3D point clouds. In addition to facilitating the computationally intensive task of processing 3D point clouds, working at the level of segments addresses the data compression requirements of real-time single-and multi-robot systems. While current methods extract descriptors for the single task of localization, SegMap leverages a data-driven descriptor in order to extract meaningful features that can also be used for reconstructing a dense 3D map of the environment and for extracting semantic information. This is particularly interesting for navigation tasks and for providing visual feedback to endusers such as robot operators, for example in search and rescue scenarios. These capabilities are demonstrated in multiple urban driving and search and rescue experiments. Our method leads to an increase of area under the ROC curve of 28.3% over current state of the art using eigenvalue based features. We also obtain very similar reconstruction capabilities to a model specifically trained for this task. The SegMap implementation is available open-source along with easy to run demonstrations at www.github.com/ethz-asl/segmap.

show abstract

“…It also needs special map encoding techniques such as elimination of both view dependency and strong gradients on TSDF. [55] [43] use tree structures, while [16] applies Hibert Maps for 3D map representation to recover the 3D shape, thus being able to produce a relatively higher resolution of 3D shape. However, their deep networks only consist of a 3D encoder and decoder, without taking advantage of adversarial learning.…”

Section: Related Workmentioning

confidence: 99%

3D Object Reconstruction from a Single Depth View with Adversarial Learning

Yang

Wen

Wang³

et al. 2017

2017 IEEE International Conference on Computer Vision Workshops (ICCVW)

176

114

View full text Add to dashboard Cite

Recent advancements in deep learning opened new opportunities for learning a high-quality 3D model from a single 2D image given sufficient training on large-scale data sets. However, the significant imbalance between available amount of images and 3D models, and the limited availability of labeled 2D image data (i.e. manually annotated pairs between images and their corresponding 3D models), severely impacts the training of most supervised deep learning methods in practice. In this paper, driven by a novel design of adversarial networks, we have developed an unsupervised learning paradigm to reconstruct 3D models from a single 2D image, which is free of manually annotated pairwise input image and its associated 3D model. Particularly, the paradigm begins with training an adaption network via autoencoder with adversarial loss, which embeds unpaired 2D synthesized image domain with real world image domain to a shared latent vector space. Then, we jointly train a 3D deconvolutional network to transform the latent vector space to the 3D object space together with the embedding process. Our experiments verify our network's robust and superior performance in handling 3D volumetric object generation from a single 2D image.Existing works on 3D object reconstruction from 2D image(s) can be broadly categorized as two of the following: traditional methods without learning; deep learning based methods. 3D reconstruction without learning. The majority of traditional reconstruction methods based on SFM or SLAM [1,2] are subject to a dense number of views, and most of them rely on the hypothesis that features can be matched across views. 2D to 3D reconstruction models such as multi-view stereo [9, 10], space carving [11], multiple moving object and large scale structure from motion [3][4][5], have all demonstrated good performance in solving the 2D to 3D reconstruction problem. However these methods require high calibrated cameras and segmentation of objects from their background, which are less applicable in practice. Deep Neural Networks in 3D visual computing. Nowadays, by generating 3D volumetric data [12], prominent deep learning models such as the deep 2D convolutional neural networks can be naturally extended to learn 3D objects. Deep learning models have proven to have strong capabilities in learning latent representative vector space of 3D objects [12]. Multi-View CNN, Conv-DAE, Voxnet, Gift, T-L embedding, 3DGAN and so on, have uncovered great potential for solving retrieval, classification, 3D reconstruction problem, etc. on [13][14][15][16][17][18].In contrast to the vast amount of research and accomplishments in the field of 3D object classification and retrieval, there are fewer research and far less accomplished results on 3D object reconstruction. Recently, researchers began to utilize 3D deconvolutional neural network to generate 3D volumetric objects from 2D images, for instance, 3D- GAN[18] and T-L embedding [17] strive to learn a latent vector space representation of 2D images, and then transform it to gene...

show abstract

Learning to Reconstruct 3D Structures for Occupancy Mapping

Cited by 17 publications

References 24 publications

SegMap: Segment-based mapping and localization using data-driven descriptors

SegMap: Segment-based mapping and localization using data-driven descriptors

SegMap: 3D Segment Mapping using Data-Driven Descriptors

3D Object Reconstruction from a Single Depth View with Adversarial Learning

Contact Info

Product

Resources

About