Dense 3D semantic mapping of indoor scenes from RGB-D images

Hermans, Alexander; Floros, George; Leibe, Bastian

doi:10.1109/icra.2014.6907236

Cited by 250 publications

(224 citation statements)

References 14 publications

Supporting

Mentioning

223

Contrasting

Order By: Relevance

“…Hermans et al [9] use a random forest classifier and a dense 2D CRF, transfer the resulting marginals into 3D and [4] out only Valentin et al [12] Häne et al [8] N/A Kundu et al [11] Hermans et al [9] Hu et al [27] Ours solve a 3D CRF to refine the predictions. Other shortcomings aside (see Tab.…”

Section: B Semantic Segmentationmentioning

confidence: 99%

“…Dense reconstructions working on a regular voxel grid [18]- [20] are limited to small volumes due to memory requirements. This has been addressed by approaches that use scalable data structures and stream data between GPU and CPU memory [21], [22], but they use Kinect-like cameras that only work indoors [9], [10]. Approaches working outdoors usually take significant time to run [4], [8], [11], [23], do not work incrementally [12] or rely on LIDAR data [24].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Incremental dense semantic stereo fusion for large-scale semantic scene reconstruction

Vineet¹,

Mikšík

Lidegaard

et al. 2015

2015 IEEE International Conference on Robotics and Automation (ICRA)

183

127

View full text Add to dashboard Cite

Abstract-Our abilities in scene understanding, which allow us to perceive the 3D structure of our surroundings and intuitively recognise the objects we see, are things that we largely take for granted, but for robots, the task of understanding large scenes quickly remains extremely challenging. Recently, scene understanding approaches based on 3D reconstruction and semantic segmentation have become popular, but existing methods either do not scale, fail outdoors, provide only sparse reconstructions or are rather slow. In this paper, we build on a recent hash-based technique for large-scale fusion and an efficient mean-field inference algorithm for densely-connected CRFs to present what to our knowledge is the first system that can perform dense, large-scale, outdoor semantic reconstruction of a scene in (near) real time. We also present a 'semantic fusion' approach that allows us to handle dynamic objects more effectively than previous approaches. We demonstrate the effectiveness of our approach on the KITTI dataset, and provide qualitative and quantitative results showing high-quality dense reconstruction and labelling of a number of scenes.

show abstract

Section: B Semantic Segmentationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Incremental dense semantic stereo fusion for large-scale semantic scene reconstruction

Vineet¹,

Mikšík

Lidegaard

et al. 2015

2015 IEEE International Conference on Robotics and Automation (ICRA)

183

127

View full text Add to dashboard Cite

show abstract

“…We use the semantic segmentation method of Husain et al [4], which is a feature learning approach similar to Eigen and Fergus [17] and Long et al [18]. Other approaches for semantic segmentation introduce hand-crafted features in their model such as gradient, colour, local binary pattern, depth gradient, spin, surface normals by Wu et al [19] and pixel value comparison and oriented gradients by Hermans et al [20]. Our method combines the final segmentation result and does not rely on any particular feature, hence it is compatible with any approach that clearly separates the object classes.…”

Section: Related Workmentioning

confidence: 99%

Semantic segmentation priors for object discovery

García

Husain

Schulz

et al. 2016

2016 23rd International Conference on Pattern Recognition (ICPR)

View full text Add to dashboard Cite

Abstract-Reliable object discovery in realistic indoor scenes is a necessity for many computer vision and service robot applications. In these scenes, semantic segmentation methods have made huge advances in recent years. Such methods can provide useful prior information for object discovery by removing false positives and by delineating object boundaries. We propose a novel method that combines bottom-up object discovery and semantic priors for producing generic object candidates in RGB-D images. We use a deep learning method for semantic segmentation to classify colour and depth superpixels into meaningful categories. Separately for each category, we use saliency to estimate the location and scale of objects, and superpixels to find their precise boundaries. Finally, object candidates of all categories are combined and ranked. We evaluate our approach on the NYU Depth V2 dataset and show that we outperform other state-of-the-art object discovery methods in terms of recall.

show abstract

“…RELATED WORK The conventional approach to semantic labeling is carried out in multiple stages [4,[15][16][17][18][19]. This involves presegmenting the scene into smaller patches followed by feature extraction and classification.…”

Section: Introductionmentioning

confidence: 99%

Combining Semantic and Geometric Features for Object Class Segmentation of Indoor Scenes

Husain

Schulz

Dellen

et al. 2017

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Abstract-Scene understanding is a necessary prerequisite for robots acting autonomously in complex environments. Low-cost RGB-D cameras such as Microsoft Kinect enabled new methods for analyzing indoor scenes and are now ubiquitously used in indoor robotics. We investigate strategies for efficient pixelwise object class labeling of indoor scenes that combine both pretrained semantic features transferred from a large color image dataset and geometric features, computed relative to the room structures, including a novel distance-from-wall feature, which encodes the proximity of scene points to a detected major wall of the room. We evaluate our approach on the popular NYU v2 dataset. Several deep learning models are tested, which are designed to exploit different characteristics of the data. This includes feature learning with two different pooling sizes. Our results indicate that combining semantic and geometric features yields significantly improved results for the task of object class segmentation.

show abstract

Dense 3D semantic mapping of indoor scenes from RGB-D images

Cited by 250 publications

References 14 publications

Incremental dense semantic stereo fusion for large-scale semantic scene reconstruction

Incremental dense semantic stereo fusion for large-scale semantic scene reconstruction

Semantic segmentation priors for object discovery

Combining Semantic and Geometric Features for Object Class Segmentation of Indoor Scenes

Contact Info

Product

Resources

About