Semi-Automatic 2D-to-3D Conversion Using Low-Rank Matrix Recovery

Abstract

Abstract-Semi-automatic 2D-to-3D conversion is a promising solution to 3D stereoscopic content creation. However, the depth continuous transition between user marked neighboring regions will be lost when user scribbles are sparse. To help solve this problem, a piecewise-continuity regularized low-rank matrix recovery method is developed. Our approach is based on the fact that a depth-map can be decomposed into a low-rank matrix and an outlier term matrix. First, an initial dense depth-map is interpolated from the user scribbles using matting Laplacian scheme under the assumption that depth-map is piecewise-continuous. Second, a piecewise-continuity constrained low-rank recovery model is developed to remove outliers which are introduced by the interpolation. Experimental comparisons with existing algorithms show that our method demonstrates significant advantage over depth continuous transition between neighboring regions.Keywords-2D-to-3D conversion, depth estimation, piecewise-continuity, low-rank, sparse interpolation IntroductionWith the development of 3D display technology, increasingly kinds of 3D electronic products, such as television, mobile phone, projector, are appearing in the ordinary people's life [1]. However, there is little 3D content to be played on these devices. Most videos and images are still in 2D. Thus, it is urgent need for 2D to 3D conversion which can generate 3D content from existing 2D images/videos.The main challenge of 2D to 3D conversion is how to retrieve the depth information from 2D images/videos which lost in the capture process. Existing 2D to 3D conversion methods can be generally divided into two categories: automatic and semiautomatic ones. The automatic conversion methods rely on different kinds of depth cues to generate depth-maps. Since the relationships between these cues and depth are nonlinear, current automatic methods usually make some global assumptions about the scene. Once the assumptions do not hold, depth errors will appear. Therefore, the accuracy of depth-maps generated by the automatic methods still can't meet the 3D display demand. 104