Light field superresolution

Abstract

Light field cameras have been recently shown to be very effective in applications such as digital refocusing and 3D reconstruction. In a single snapshot these cameras provide a sample of the light field of a scene by trading off spatial resolution with angular resolution. Current methods produce images at a resolution that is much lower than that of traditional imaging devices. However, by explicitly modeling the image formation process and incorporating priors such as Lambertianity and texture statistics, these types of images can be reconstructed at a higher resolution. We formulate this method in a variational Bayesian framework and perform the reconstruction of both the surface of the scene and the (superresolved) light field. The method is demonstrated on both synthetic and real images captured with our light-field camera prototype.

An overview of computational photography

“…High resolution images can be obtained via professional cameras, but these cameras are usually costly. The spatial resolution can be enhanced computationally: e.g., Wilburn et al [84] captured high resolution images using a large camera array, while Bishop et al [119] raised image resolution by introducing prior constraints, and Landolt et al [120] attained the same goal by introducing mechanical vibrations (named jittering) to the image sensor; Wang et al [121] and Ben-Ezra [122] designed large format cameras for high resolution image capture. One newly proposed work is the giga-pixel imaging proposed by Cossairt et al [123], who designed a compact architecture composed of a ball lens shared by several small planar sensors, and a post-capture image processing stage; one example image captured by their prototypes is shown in Figure 4.…”

Single Image Blind Deconvolution with Higher-Order Texture Statistics

“…One technique is to obtain multiple viewpoints by capturing multiple coded images [8,13,22] or by capturing a single image by using a plenoptic camera [9,6,23,24]. These methods however, exploit multiple images or require a more costly optical design (e.g., a calibrated microlens array).…”

“…Perhaps one of the most remarkable results is to have shown that it is possible to extend the depth of field of a camera by modifying the camera optical response [1,2,3,4,5,6,7]. Moreover, techniques based on applying a mask at the lens aperture have demonstrated the ability to recover a coarse depth of the scene [4,5,8].…”

Towards Motion Aware Light Field Video for Dynamic Scenes

“…Recently, several LF super-resolution algorithms have been proposed to recover the lost resolution [12,4]. The Plenoptic2.0 camera [12] recovers the lost resolution by placing the microlens array at a different location compared to the original design [24].…”

“…Thus, it is clear that there exists a resolution barrier ( Figure 1) for capturing high resolution LF video. Recently, a series of approaches such as Plenoptic 2.0 camera [12] and LF super-resolution techniques [4] have begun breaking this resolution barrier. Simulated scene with moving butterfly and beetle and static grass.…”