Depth Enhancement via Low-Rank Matrix Completion

Si, Lu; Ren, Xipei; Liu, Feng

doi:10.1109/cvpr.2014.433

Cited by 161 publications

(133 citation statements)

References 34 publications

Supporting

Mentioning

133

Contrasting

Order By: Relevance

“…Four representative test images, RGBZ_01, RGBZ_03, RGBZ_05, and RGBZ_07, from the RGBZ dataset [23] are used to evaluate the performance. e proposed method is compared with several state-of-the-art [4].…”

Section: Methodsmentioning

confidence: 99%

Inaccuracy-Tolerant Sparse-to-Dense Depth Propagation for Semiautomatic 2D-to-3D Conversion

Yuan

2018

Journal of Electrical and Computer Engineering

View full text Add to dashboard Cite

Current semiautomatic 2D-to-3D methods assume that user input is perfectly accurate. However, it is difficult to get 100% accurate user scribbles and even small errors in the input will degrade the conversion quality. is paper addresses the issue with scribble confidence that considers color differences between labeled pixels and their neighbors. First, it counts the number of neighbors which have similar and different color values for each labeled pixels, respectively. e ratio between these two numbers at each labeled pixel is regarded as its scribble confidence. Second, the sparse-to-dense depth conversion is formulated as a confident optimization problem by introducing a confident weighting data cost term and the local and knearest depth consistent regularization terms. Finally, the dense depth-map is obtained by solving sparse linear equations. e proposed approach is compared with existing methods on several representative images. e experimental results demonstrate that the proposed method can tolerate some errors from use input and can reduce depth-map artifacts caused by inaccurate user input.

show abstract

Section: Methodsmentioning

confidence: 99%

Inaccuracy-Tolerant Sparse-to-Dense Depth Propagation for Semiautomatic 2D-to-3D Conversion

Yuan

2018

Journal of Electrical and Computer Engineering

View full text Add to dashboard Cite

show abstract

“…Inspired by the recent success of low-rank methods application in image processing, e.g., colorization [2], image restoration [16], texture repairing [17] and etc., we develop a depth estimation approach combining matting Laplacian and low-rank method which captures the continuous depth changes while removing texture details intro-http://www.i-joe.org duced by matting Laplacian scheme. We notice that Lu et al [18] also apply low-rank constraints on depth enhancement. The differences between our and their method are: (1) Lu et al [18] assume RGB-D patches lie in a low-dimensional subspace and we apply low-rank regularization to the whole depth-map; (2) the focus of [18] is depthmap completion and our problem is sparse-to-dense depth propagation in semiautomatic 2D to 3D conversion.…”

Section: Related Workmentioning

confidence: 99%

“…We notice that Lu et al [18] also apply low-rank constraints on depth enhancement. The differences between our and their method are: (1) Lu et al [18] assume RGB-D patches lie in a low-dimensional subspace and we apply low-rank regularization to the whole depth-map; (2) the focus of [18] is depthmap completion and our problem is sparse-to-dense depth propagation in semiautomatic 2D to 3D conversion.…”

Section: Related Workmentioning

confidence: 99%

Semi-Automatic 2D-to-3D Conversion Using Low-Rank Matrix Recovery

Yuan

2018

Int. J. Onl. Eng.

View full text Add to dashboard Cite

Abstract-Semi-automatic 2D-to-3D conversion is a promising solution to 3D stereoscopic content creation. However, the depth continuous transition between user marked neighboring regions will be lost when user scribbles are sparse. To help solve this problem, a piecewise-continuity regularized low-rank matrix recovery method is developed. Our approach is based on the fact that a depth-map can be decomposed into a low-rank matrix and an outlier term matrix. First, an initial dense depth-map is interpolated from the user scribbles using matting Laplacian scheme under the assumption that depth-map is piecewise-continuous. Second, a piecewise-continuity constrained low-rank recovery model is developed to remove outliers which are introduced by the interpolation. Experimental comparisons with existing algorithms show that our method demonstrates significant advantage over depth continuous transition between neighboring regions.Keywords-2D-to-3D conversion, depth estimation, piecewise-continuity, low-rank, sparse interpolation IntroductionWith the development of 3D display technology, increasingly kinds of 3D electronic products, such as television, mobile phone, projector, are appearing in the ordinary people's life [1]. However, there is little 3D content to be played on these devices. Most videos and images are still in 2D. Thus, it is urgent need for 2D to 3D conversion which can generate 3D content from existing 2D images/videos.The main challenge of 2D to 3D conversion is how to retrieve the depth information from 2D images/videos which lost in the capture process. Existing 2D to 3D conversion methods can be generally divided into two categories: automatic and semiautomatic ones. The automatic conversion methods rely on different kinds of depth cues to generate depth-maps. Since the relationships between these cues and depth are nonlinear, current automatic methods usually make some global assumptions about the scene. Once the assumptions do not hold, depth errors will appear. Therefore, the accuracy of depth-maps generated by the automatic methods still can't meet the 3D display demand. 104

show abstract

“…This is typically achieved via Markov Random Fields [6,21,7], bilateral filtering [22], layered representations [23], patch-based approaches [10,11], or depth transfer [8,9]. These approaches, however, inherently assume to have access to regularly-spaced depth measurements, and thus cannot handle large holes in depth maps.…”

Section: Related Workmentioning

confidence: 99%

“…For instance, depth super-resolution [6][7][8][9][10][11] tackles the sparseness issue and attempts to densify the observed depth data. Typically, however, existing methods assume that the measurements are regularly spaced, and are thus ill-suited to handle large holes.…”

Section: Introductionmentioning

confidence: 99%

Building Scene Models by Completing and Hallucinating Depth and Semantics

Liu

Salzmann

2016

Computer Vision – ECCV 2016

View full text Add to dashboard Cite

Abstract. Building 3D scene models has been a longstanding goal of computer vision. The great progress in depth sensors brings us one step closer to achieving this in a single shot. However, depth sensors still produce imperfect measurements that are sparse and contain holes. While depth completion aims at tackling this issue, it ignores the fact that some regions of the scene are occluded by the foreground objects. Building a scene model would therefore require to hallucinate the depth behind these objects. In contrast with existing methods that either rely on manual input, or focus on the indoor scenario, we introduce a fully-automatic method to jointly complete and hallucinate depth and semantics in challenging outdoor scenes. To this end, we develop a two-layer model representing both the visible information and the hidden one. At the heart of our approach lies a formulation based on the Mumford-Shah functional, for which we derive an effective optimization strategy. Our experiments evidence that our approach can accurately fill the large holes in the input depth maps, segment the different kinds of objects in the scene, and hallucinate the depth and semantics behind the foreground objects.

show abstract

Depth Enhancement via Low-Rank Matrix Completion

Abstract: Abstract

Cited by 161 publications

References 34 publications

Inaccuracy-Tolerant Sparse-to-Dense Depth Propagation for Semiautomatic 2D-to-3D Conversion

Inaccuracy-Tolerant Sparse-to-Dense Depth Propagation for Semiautomatic 2D-to-3D Conversion

Semi-Automatic 2D-to-3D Conversion Using Low-Rank Matrix Recovery

Building Scene Models by Completing and Hallucinating Depth and Semantics

Contact Info

Product

Resources

About