Reconstructing Interlaced High-Dynamic-Range Video Using Joint Learning

Abstract

For extending the dynamic range of video, it is a common practice to capture multiple frames sequentially with different exposures and combine them to extend the dynamic range of each video frame. However, this approach results in typical ghosting artifacts due to fast and complex motion in nature. As an alternative, video imaging with interlaced exposures has been introduced to extend the dynamic range. However, the interlaced approach has been hindered by jaggy artifacts and sensor noise, leading to concerns over image quality. In this paper, we propose a data-driven approach for jointly solving two specific problems of deinterlacing and denoising that arise in interlaced video imaging with different exposures. First, we solve the deinterlacing problem using joint dictionary learning via sparse coding. Since partial information of detail in differently exposed rows is often available via interlacing, we make use of the information to reconstruct details of the extended dynamic range from the interlaced video input. Second, we jointly solve the denoising problem by tailoring sparse coding to better handle additive noise in low-/high-exposure rows, and also adopt multiscale homography flow to temporal sequences for denoising. We anticipate that the proposed method will allow for concurrent capture of higher dynamic range video frames without suffering from ghosting artifacts. We demonstrate the advantages of our interlaced video imaging compared with the state-of-the-art high-dynamic-range video methods.

Deep Joint Deinterlacing and Denoising for Single Shot Dual-ISO HDR Reconstruction

“…1 shows that how they prepare the exposures for their joint-learning-based reconstruction. If the pixels…”

Computational Photography : High Dynamic Rangeand Light Fields

“…by utilizing the dual-ISO technique, and reconstruct the interlaced frames using our unified framework for HDR reconstruction with three-dimensional kernels including both spatial and temporal information. The sparse representation can also be utilized for the reconstruction of HDR video from dual-ISO frames, similar to [262]. Another interesting research direction for the HDR reconstruction method is to explore the similarity of nonlocal regions of the image and incorporate that in the reconstruction algorithm.…”

Deep HDR Video from Sequences with Alternating Exposures

“…Over the past decades, many powerful approaches have been developed to produce still HDR images from sequences with different exposures [DM97,SKY * A large number of approaches propose to capture HDR images and videos through coded per-pixel [NM00,NN02,SHG * 16] or scanline exposure/ISO [HST * 14, HKU15,CBK17], and generate the results using appropriate reconstruction techniques. These methods can to work on input images with varying ISO, which can be captured with off-the-shelf cameras.…”