Simultaneous Super-Resolution of Depth and Images Using a Single Camera

Abstract

In this paper, we propose a convex optimization framework for simultaneous estimation of super-resolved depth map and images from a single moving camera. The pixel measurement error in 3D reconstruction is directly related to the resolution of the images at hand. In turn, even a small measurement error can cause significant errors in reconstructing 3D scene structure or camera pose. Therefore, enhancing image resolution can be an effective solution for securing the accuracy as well as the resolution of 3D reconstruction. In the proposed method, depth map estimation and image super-resolution are formulated in a single energy minimization framework with a convex function and solved efficiently by a first-order primal-dual algorithm. Explicit inter-frame pixel correspondences are not required for our super-resolution procedure, thus we can avoid a huge computation time and obtain improved depth map in the accuracy and resolution as well as highresolution images with reasonable time. The superiority of our algorithm is demonstrated by presenting the improved depth map accuracy, image super-resolution results, and camera pose estimation.

Data-Driven Tight Frame for Multi-channel Images and Its Application to Joint Color-Depth Image Reconstruction

“…[35] proposes to approximate piecewise smooth functions in frame domain with sparse representation, and a tight frame based scheme is developed to recover depth image from noise and outliers corrupted data. [30] presents a convex optimization framework to simultaneously estimate super-resolution of both depth images and color images, in which the problem is formulated into a single energy minimization with a convex function. Similarly, [18] proposes to utilize the depth cues from the induced optical-flow to enhance the depth images captured by a moving device.…”

“…Furthermore, the RGB-D camera may produce a severely noised color image under some bad illumination conditions. Thus the enhancement of depth image and RGB-D image is receiving increasing research interest [18,24,30,35]. We will employ the proposed multi-channel data-driven tight frame for the joint task of restoration of the depth image and noise removal of the color image.…”

A Survey -- Super Resolution Techniques for Multiple, Single, and Stereo Images

“…They up-sample the depth map to the desired resolution (same factor as used at encoder), then performing the forward mapping from left to right followed by median filtering to obtain the HR depth map. Lee and Lee [17] found that the depth reconstruction and super resolution of LR stereo images are interrelated, so they proposed the problem of super resolving intensity and depth under a single framework and solved using first-order primal dual algorithm.…”

Joint Estimation of Camera Pose, Depth, Deblurring, and Super-Resolution from a Blurred Image Sequence

“…Low-resolution (LR) input images also often affect stereo matching accuracy [5,15], because low-quality cameras are frequently used considering the limitations in cost or space for some applications. However, even in a highresolution (HR) image, the actual scene resolution is spatially uneven and dependent on depth because of the perspective projection.…”