Sliding Shapes for 3D Object Detection in Depth Images

Song, Shuran; Xiao, Jianxiong

doi:10.1007/978-3-319-10599-4_41

Cited by 332 publications

(295 citation statements)

References 81 publications

Supporting

Mentioning

294

Contrasting

Unclassified

Order By: Relevance

“…As an empirical observation, we estimate that the gap size should be roughly equal to half the average cluster size for an optimal reconstruction. Note that these cluster sizes are much larger than the ones usually found in the literature for similar tasks, such as object detection and scene reconstruction [34,20,30].…”

Section: Resultsmentioning

confidence: 73%

Learning to Reconstruct 3D Structures for Occupancy Mapping

Guizilini

Ramos

2017

Robotics: Science and Systems XIII

View full text Add to dashboard Cite

Abstract-Real world scenarios contain many structural patterns that, if appropriately extracted and modeled, can be used to reduce problems associated with sensor failure and occlusions, while improving planning methods in tasks such as navigation and grasping. This paper devises a novel unsupervised procedure that is able to learn 3D structures from unorganized point clouds as occupancy maps. Our framework enables the learning of unique and arbitrarily complex features using a Bayesian Convolutional Variational Auto-Encoder that compresses local information into a latent low-dimensional representation and then decodes it back in order to reconstruct the original scene. This reconstructive model is trained on features obtained automatically from a wide variety of scenarios to improve its generalization and interpolative powers. We show that the proposed framework is able to recover partially missing structures and reason over occlusion with high accuracy, while maintaining a detailed reconstruction of observed areas. To seamlessly combine this localized feature information into a single global structure, we employ a Hilbert Map, recently proposed as a robust and efficient occupancy mapping technique. Experimental tests are conducted in large-scale 2D and 3D datasets, and a study on the impact of various accuracy/speed trade-offs is provided to assess the limits of the proposed framework.

show abstract

Section: Resultsmentioning

confidence: 73%

Learning to Reconstruct 3D Structures for Occupancy Mapping

Guizilini

Ramos

2017

Robotics: Science and Systems XIII

View full text Add to dashboard Cite

show abstract

“…Barron et al [6] leverage the depth for intrinsic image decomposition of natural scenes. Song and Xiao [34] show that depth can avoid many difficulties for object detection from RGB images, such as variations in illumination, shape, and viewpoint. In particular, they adopt the Truncated Signed Distance Function (TSDF) as a cue for self-occlusion.…”

Section: Previous Workmentioning

confidence: 99%

“…As consumer depth sensors become widely adopted, e.g. Microsoft Kinect, the depth information has brought breakthroughs to several vision tasks, such as pose estimation [31], intrinsic image decomposition [6], and object detection [34]. The idea of leveraging depth cues to allow monocular scene flow estimation has also received increasing attention.…”

Section: Introductionmentioning

confidence: 99%

Layered RGBD scene flow estimation

Sun

Sudderth

Pfister

2015

2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Our layered approach can handle multiple moving objects and reliably estimate their motion in occlusion regions. Our key observation is that depth provides the depth ordering information, thereby solving a computational bottleneck for previous RGB layered methods (please see Figure 6 for our detected occlusions and the estimated motion by the recent semi-rigid scene flow (SRSF) method [27]). AbstractAs consumer depth sensors become widely available, estimating scene flow from RGBD sequences has received increasing attention. Although the depth information allows the recovery of 3D motion from a single view, it poses new challenges. In particular, depth boundaries are not wellaligned with RGB image edges and therefore not reliable cues to localize 2D motion boundaries. In addition, methods that extend the 2D optical flow formulation to 3D still produce large errors in occlusion regions. To better use depth for occlusion reasoning, we propose a layered RGBD scene flow method that jointly solves for the scene segmentation and the motion. Our key observation is that the noisy depth is sufficient to decide the depth ordering of layers, thereby avoiding a computational bottleneck for RGB layered methods. Furthermore, the depth enables us to estimate a per-layer 3D rigid motion to constrain the motion of each layer. Experimental results on both the Middlebury and real-world sequences demonstrate the effectiveness of the layered approach for RGBD scene flow estimation.

show abstract

“…For example, [18,17] use objects from Google's 3D Warehouse to train an object detection system for 3D point clouds collected by robots navigating through urban and indoor environments. [28] uses 3D CAD models as templates for a sliding window search. Shen et al [25] use a database of segmented models and assembly-based modeling algorithms to reconstruct novel 3D models.…”

Section: Related Workmentioning

confidence: 99%

Learning Hierarchical Semantic Segmentations of LIDAR Data

Dohan

Matejek

Funkhouser

2015

2015 International Conference on 3D Vision

View full text Add to dashboard Cite

This paper investigates a method for semantic segmentation of small objects in terrestrial LIDAR scans in urban environments. The core research contribution is a hierarchical segmentation algorithm where potential merges between segments are prioritized by a learned affinity function and constrained to occur only if they achieve a significantly high object classification probability. This approach provides a way to integrate a learned shape-prior (the object classifier) into a search for the best semantic segmentation in a fast and practical algorithm. Experiments with LIDAR scans collected by Google Street View cars throughout ∼100 city blocks of New York City show that the algorithm provides better segmentations and classifications than simple alternatives for cars, vans, traffic lights, and street lights.

show abstract

Sliding Shapes for 3D Object Detection in Depth Images

Cited by 332 publications

References 81 publications

Learning to Reconstruct 3D Structures for Occupancy Mapping

Learning to Reconstruct 3D Structures for Occupancy Mapping

Layered RGBD scene flow estimation

Learning Hierarchical Semantic Segmentations of LIDAR Data

Contact Info

Product

Resources

About