The Second Workshop on 3D Reconstruction Meets Semantics: Challenge Results Discussion

Tyleček, Radim; Sattler, Torsten; Le, Hoang-An; Brox, Thomas; Pollefeys, Marc; Fisher, Robert B.; Gevers, Theo

doi:10.1007/978-3-030-11015-4_48

Cited by 11 publications

(14 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Aiming at showing that our approach generalizes well to various types of environments, we run a series of experiments on three completely different datasets for 3D semantic segmentation containing image sequences: ScanNet [28], a RGB-D video dataset established as one of the standard 3D segmentation benchmarks; the 3DRMS Challenge Dataset [29], a challenging outdoor dataset, featuring image sequences captured from virtual and real drives through semantically-rich garden scenes; and the Aerial Depth Dataset [1], a photorealistic aerial dataset that exhibits more challenging scenarios than the established indoor and car driving datasets. In all of them, ground truth 2D semantic annotations and camera poses, as well as semantically annotated 3D point clouds depicting the area of each sequence are provided for evaluation.…”

Section: Methodsmentioning

confidence: 99%

“…To evaluate the performance of our approach on other challenging, but completely different environments, we also run experiments on the real-world validation sequence of the 2018 3DRMS Challenge [29] as well as on one sequence extracted from the Aerial Depth Dataset [1]. Here, given a set of RGB images with a known camera pose, the goal is to label the provided 3D point cloud of the scene.…”

Section: B Evaluation On Outdoor Datasetsmentioning

confidence: 99%

“…Again, the proposed Diffuser exhibits significant improvement with respect to the Bayesian fusion approach by providing a more accurate Fig. 4: Qualitative segmentation results on the real-world validation sequence of the 3DRMS Challenge dataset [29]. For evaluation purposes, the classes considered by the original dataset are associated with the MSeg universal classes [2] as follows: grass and ground are mapped to terrain, while hedge, topiary, rose and tree are mapped to vegetation.…”

Section: B Evaluation On Outdoor Datasetsmentioning

confidence: 99%

See 2 more Smart Citations

Diffuser: Multi-View 2D-to-3D Label Diffusion for Semantic Scene Segmentation

Mascaro

Teixeira

Chli

2021

2021 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Semantic 3D scene understanding is a fundamental problem in computer vision and robotics. Despite recent advances in deep learning, its application to multi-domain 3D semantic segmentation typically suffers from the lack of extensive enough annotated 3D datasets. On the contrary, 2D neural networks benefit from existing large amounts of training data and can be applied to a wider variety of environments, sometimes even without need for retraining. In this paper, we present 'Diffuser', a novel and efficient multi-view fusion framework that leverages 2D semantic segmentation of multiple image views of a scene to produce a consistent and refined 3D segmentation. We formulate the 3D segmentation task as a transductive label diffusion problem on a graph, where multi-view and 3D geometric properties are used to propagate semantic labels from the 2D image space to the 3D map. Experiments conducted on indoor and outdoor challenging datasets demonstrate the versatility of our approach, as well as its effectiveness for both global 3D scene labeling and single RGB-D frame segmentation. Furthermore, we show a significant increase in 3D segmentation accuracy compared to probabilistic fusion methods employed in several state-of-theart multi-view approaches, with little computational overhead.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: B Evaluation On Outdoor Datasetsmentioning

confidence: 99%

Section: B Evaluation On Outdoor Datasetsmentioning

confidence: 99%

See 1 more Smart Citation

Diffuser: Multi-View 2D-to-3D Label Diffusion for Semantic Scene Segmentation

Mascaro

Teixeira

Chli

2021

2021 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

show abstract

“…There is no reference value or method for the KITTI scene reconstruction task regarding 3D point clouds. Therefore, we propose to measure performances using the completeness and accuracy criteria from [22], and also propose the relative accuracy. All 3 measures are defined as follows:…”

Section: Evaluation Metricsmentioning

confidence: 99%

Pix2Point: Learning Outdoor 3D Using Sparse Point Clouds and Optimal Transport

Leroy¹,

Trouvé-Peloux²,

Champagnat³

et al. 2021

Preprint

View full text Add to dashboard Cite

Good quality reconstruction and comprehension of a scene rely on 3D estimation methods. The 3D information was usually obtained from images by stereophotogrammetry, but deep learning has recently provided us with excellent results for monocular depth estimation. Building up a sufficiently large and rich training dataset to achieve these results requires onerous processing. In this paper, we address the problem of learning outdoor 3D point cloud from monocular data using a sparse ground-truth dataset. We propose Pix2Point, a deep learning-based approach for monocular 3D point cloud prediction, able to deal with complete and challenging outdoor scenes. Our method relies on a 2D-3D hybrid neural network architecture, and a supervised end-to-end minimisation of an optimal transport divergence between point clouds. We show that, when trained on sparse point clouds, our simple promising approach achieves a better coverage of 3D outdoor scenes than efficient monocular depth methods.

show abstract

“…

…”

mentioning

confidence: 99%

Technical Report: Co-learning of geometry and semantics for online 3D mapping

Carvalho¹,

Ferrera²,

Boulch³

et al. 2019

Preprint

View full text Add to dashboard Cite

This paper is a technical report about our submission for the ECCV 2018 3DRMS Workshop Challenge on Semantic 3D Reconstruction [1]. In this paper, we address 3D semantic reconstruction for autonomous navigation using co-learning of depth map and semantic segmentation. The core of our pipeline is a deep multi-task neural network which tightly refines depth and also produces accurate semantic segmentation maps. Its inputs are an image and a raw depth map produced from a pair of images by standard stereo vision. The resulting semantic 3D point clouds are then merged in order to create a consistent 3D mesh, in turn used to produce dense semantic 3D reconstruction maps. The performances of each step of the proposed method are evaluated on the dataset and multiple tasks of the 3DRMS Challenge, and repeatedly surpass state-of-the-art approaches. Fig. 1. Pipeline for generating geometric and semantic 3D reconstruction maps.

show abstract

The Second Workshop on 3D Reconstruction Meets Semantics: Challenge Results Discussion

Cited by 11 publications

References 14 publications

Diffuser: Multi-View 2D-to-3D Label Diffusion for Semantic Scene Segmentation

Diffuser: Multi-View 2D-to-3D Label Diffusion for Semantic Scene Segmentation

Pix2Point: Learning Outdoor 3D Using Sparse Point Clouds and Optimal Transport

Technical Report: Co-learning of geometry and semantics for online 3D mapping

Contact Info

Product

Resources

About