Deep high dynamic range imaging of dynamic scenes

Abstract

Producing a high dynamic range (HDR) image from a set of images with different exposures is a challenging process for dynamic scenes. A category of existing techniques first register the input images to a reference image and then merge the aligned images into an HDR image. However, the artifacts of the registration usually appear as ghosting and tearing in the final HDR images. In this paper, we propose a learning-based approach to address this problem for dynamic scenes. We use a convolutional neural network (CNN) as our learning model and present and compare three different system architectures to model the HDR merge process. Furthermore, we create a large dataset of input LDR images and their corresponding ground truth HDR images to train our system. We demonstrate the performance of our system by producing high-quality HDR images from a set of three LDR images. Experimental results show that our method consistently produces better results than several state-of-the-art approaches on challenging scenes.

Deep HDR Video from Sequences with Alternating Exposures

“…Comparing to the network architecture, as proposed by Kalantari and Ramamoorthi [KR17], our encoder‐decoder architecture produces results with fewer discoloration and objectionable artifacts. Note that, Kalantari's network is retrained on the HDR video data to have a fair comparison.…”

“…As shown in the insets, both aligned neighboring images, Z̃ i –1 , i and Z̃ i +1 , i , have registration artifacts on the grill cover. Therefore, using only the aligned images, as proposed by Kalantari and Ramamoorthi [KR17], we are not able to properly reconstruct the missing content. However, as indicated by the green arrow, the grill cover is artifact‐free in one of the neighboring images, Z i −1 .…”

“…3) and learning‐based (Fig. 4) optical flow approaches. We apply necessary changes to the input and architecture of the merge network by Kalantari et al [KR17] (Sec. 3.3) to significantly improve the quality of the results (Figs.…”

The high dynamic range imaging pipeline : Tone-mapping, distribution, and single-exposure reconstruction

“…There is a large body of work on HDR image registration and deghosting, facilitating HDR exposure bracketing of dynamic scenes. For example, for per-pixel registration optical flow can be used [108,125,280], or patch-based approaches [118,224]. For a thorough survey and categorization, we refer to the state-of-the-art report by Tursun et al [244].…”

“…These make use of CNNs for a variety of problems related to HDR imaging, demonstrating various degrees of improvement over previous work. For example, there are CNNs for HDR reconstruction from multiple exposures in separate images [106,125,266] and from single-shot, spatially varying, exposures [9]. Other techniques attempt to estimate outdoor [115] and indoor [96] illumination maps from conventional LDR images.…”