Learning to Segment and Track in RGBD

Teichman, Alex; Lussier, Jake T.; Thrun, Sebastian

doi:10.1109/tase.2013.2264286

Cited by 29 publications

(13 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other multitask vision networks [3,12,34] also commonly use depth as a complementary output to benefit overall network performance. Using depth as an explicit input is also common in computer vision, with plentiful applications in tracking [30,33], SLAM systems [13,36] and 3d reconstruction/detection [7,19]. There is clearly a pressing demand for high-quality depth maps, but current depth hardware solutions are power-hungry, have severe range limitations [8], and the current traditional depth estimation methods [4,17] fail to achieve the accuracies necessary to supersede such hardware.…”

Section: Related Workmentioning

confidence: 99%

Estimating Depth from RGB and Sparse Sensing

Zhao

Badrinarayanan

Drozdov

et al. 2018

Lecture Notes in Computer Science

105

View full text Add to dashboard Cite

We present a deep model that can accurately produce dense depth maps given an RGB image with known depth at a very sparse set of pixels. The model works simultaneously for both indoor/outdoor scenes and produces state-of-the-art dense depth maps at nearly realtime speeds on both the NYUv2 and KITTI datasets. We surpass the state-of-the-art for monocular depth estimation even with depth values for only 1 out of every ∼ 10000 image pixels, and we outperform other sparse-to-dense depth methods at all sparsity levels. With depth values for 1/256 of the image pixels, we achieve a mean error of less than 1% of actual depth on indoor scenes, comparable to the performance of consumer-grade depth sensor hardware. Our experiments demonstrate that it would indeed be possible to efficiently transform sparse depth measurements obtained using e.g. lower-power depth sensors or SLAM systems into high-quality dense depth maps.

show abstract

Section: Related Workmentioning

confidence: 99%

Estimating Depth from RGB and Sparse Sensing

Zhao

Badrinarayanan

Drozdov

et al. 2018

Lecture Notes in Computer Science

105

View full text Add to dashboard Cite

show abstract

“…Scene modeling is performed by analyzing support relationships of the regions [18] or contextual modeling of both super-pixel MRFs and paths in segmentation trees [13]. Temporal evolution of the 3D point cloud has been considered in cases where a learned 3D model of the segmented object is available, such as in the simultaneous segmentation and pose tracking approach of [12], the rigid object tracking approach of [15], or the segmentationtracking method of an arbitrary untrained object in [21].…”

Section: Related Workmentioning

confidence: 99%

Efficient Hierarchical Graph-Based Segmentation of RGBD Videos

Hickson

Birchfield

Essa

et al. 2014

2014 IEEE Conference on Computer Vision and Pattern Recognition

View full text Add to dashboard Cite

We present an efficient and scalable algorithm for segmenting 3D RGBD point clouds by combining depth, color, and temporal information using a multistage, hierarchical graph-based approach. Our algorithm processes a moving window over several point clouds to group similar regions over a graph, resulting in an initial over-segmentation. These regions are then merged to yield a dendrogram using agglomerative clustering via a minimum spanning tree algorithm. Bipartite graph matching at a given level of the hierarchical tree yields the final segmentation of the point clouds by maintaining region identities over arbitrarily long periods of time. We show that a multistage segmentation with depth then color yields better results than a linear combination of depth and color. Due to its incremental processing, our algorithm can process videos of any length and in a streaming pipeline. The algorithm's ability to produce robust, efficient segmentation is demonstrated with numerous experimental results on challenging sequences from our own as well as public RGBD data sets.

show abstract

“…Our segmentation method is based on the method described in [10] that uses a hierarchical graph-based segmentation based on [7]. Although there are other methods that segment 3D data such as [25,29,1,29,27,19] that could have been used, we decided on [10] based off it's impressive results and open source library.…”

Section: D/4d Segmentationmentioning

confidence: 99%

Semantic Instance Labeling Leveraging Hierarchical Segmentation

Hickson

Essa

Christensen

2015

2015 IEEE Winter Conference on Applications of Computer Vision

View full text Add to dashboard Cite

Most of the approaches for indoor RGBD semantic labeling focus on using pixels or superpixels to train a classifier. In this paper, we implement a higher level segmentation using a hierarchy of superpixels to obtain a better segmentation for training our classifier. By focusing on meaningful segments that conform more directly to objects, regardless of size, we train a random forest of decision trees as a classifier using simple features such as the 3D size, LAB color histogram, width, height, and shape as specified by a histogram of surface normals. We test our method on the NYU V2 depth dataset, a challenging dataset of cluttered indoor environments. Our experiments using the NYU V2 depth dataset show that our method achieves state of the art results on both a general semantic labeling introduced by the dataset (floor, structure, furniture, and objects) and a more object specific semantic labeling. We show that training a classifier on a segmentation from a hierarchy of super pixels yields better results than training directly on super pixels, patches, or pixels as in previous work.

show abstract

Learning to Segment and Track in RGBD

Cited by 29 publications

References 22 publications

Estimating Depth from RGB and Sparse Sensing

Estimating Depth from RGB and Sparse Sensing

Efficient Hierarchical Graph-Based Segmentation of RGBD Videos

Semantic Instance Labeling Leveraging Hierarchical Segmentation

Contact Info

Product

Resources

About