Adaptive dualISO HDR reconstruction

Abstract

With the development of modern image sensors enabling flexible image acquisition, single shot high dynamic range (HDR) imaging is becoming increasingly popular. In this work, we capture single shot HDR images using an imaging sensor with spatially varying gain/ISO. This allows all incoming photons to be used in the imaging. Previous methods on single shot HDR capture use spatially varying neutral density (ND) filters which lead to wasting incoming light. The main technical contribution in this work is an extension of previous HDR reconstruction approaches for single shot HDR imaging based on local polynomial approximations (Kronander et al., Unified HDR reconstruction from raw CFA data, 2013; Hajisharif et al., HDR reconstruction for alternating gain (ISO) sensor readout, 2014). Using a sensor noise model, these works deploy a statistically informed filtering operation to reconstruct HDR pixel values. However, instead of using a fixed filter size, we introduce two novel algorithms for adaptive filter kernel selection. Unlike a previous work, using adaptive filter kernels (Signal Process Image Commun 29(2):203-215, 2014), our algorithms are based on analyzing the model fit and the expected statistical deviation of the estimate based on the sensor noise model. Using an iterative procedure, we can then adapt the filter kernel according to the image structure and the statistical image noise. Experimental results show that the proposed filter de-noises the noisy image carefully while well preserving the important image features such as edges and corners, outperforming previous methods. To demonstrate the robustness of our approach, we have exploited input images from raw sensor data using a commercial off-the-shelf camera. To further analyze our algorithm, we have also implemented a camera simulator to evaluate different gain patterns and noise properties of the sensor.

The high dynamic range imaging pipeline : Tone-mapping, distribution, and single-exposure reconstruction

“…Other techniques for accomplishing spatially varying exposures include per-row modification of the sensor readout. This can be done in order to get rows of different exposure [104] or gain [109,110,245] within the same shot. There are also more unconventional techniques for encoding highlight information in an LDR image.…”

Computational Photography : High Dynamic Rangeand Light Fields

“…For simplicity, we consider that a i is constant for all pixels. The readout noise parameters, e.g., mean, µ R , and the variance, σ 2 R , are estimated by capturing a set of dark images, {b i }, using the same camera settings, (i.e. gain, g, and exposure time, t), with the lens cap on, such that no light rays arrive at the sensor.…”

Deep HDR Video from Sequences with Alternating Exposures

“…Over the past decades, many powerful approaches have been developed to produce still HDR images from sequences with different exposures [DM97,SKY * A large number of approaches propose to capture HDR images and videos through coded per-pixel [NM00,NN02,SHG * 16] or scanline exposure/ISO [HST * 14, HKU15,CBK17], and generate the results using appropriate reconstruction techniques. These methods can to work on input images with varying ISO, which can be captured with off-the-shelf cameras.…”