Exposure Stacks of Live Scenes with Hand-Held Cameras

Abstract

Abstract. Many computational photography applications require the user to take multiple pictures of the same scene with different camera settings. While this allows to capture more information about the scene than what is possible with a single image, the approach is limited by the requirement that the images be perfectly registered. In a typical scenario the camera is hand-held and is therefore prone to moving during the capture of an image burst, while the scene is likely to contain moving objects. Combining such images without careful registration introduces annoying artifacts in the final image. This paper presents a method to register exposure stacks in the presence of both camera motion and scene changes. Our approach warps and modifies the content of the images in the stack to match that of a reference image. Even in the presence of large, highly non-rigid displacements we show that the images are correctly registered to the reference.

HDR Deghosting: How to Deal with Saturation?

“…As mentioned in Section 1, we are only aware of four methods that attempt to address the general case of camera motion and scene changes at the same time [13,30,11,24]. All the fused results were generated using the method by Mertens et al [20], with the exception of Figure 6, which was tonemapped with the method by Mantiuk et al [19] to allow for a fair comparison with the method by Sen et al Figures 3 and 4 show results sensibly better than Zimmer et al and Kang et al respectively.When the reference image is reasonably well-exposed everywhere, our method produces very similar results as Hu et al However, when part of the reference is saturated, as in Figure 5, Hu et al discard valuable information from the shorter exposure (first row, middle image); our method, on the other hand, successfully captures all the available information in the synthesized latent image (second row, middle image).…”

“…To maximize the applicability of our algorithm, we do not want to limit its scope to RAW (linear) images. Image signal processors (ISP) apply various non-linear transformations to the almost-linear pixel values; these highly non-linear transformations are usually much more sophisticated than simple gamma compressions, and they sometimes even depend on the image content [11,15], making it difficult or even impossible to invert the transformations. Hence, instead of linearizing the input images, we take inspiration from the energy definition by Darabi et al [5], but we account for a generic intensity mapping function τ :…”

“…Zimmer et al [30] propose to compute the optical flow in the gradient domain which is assumed to remain constant as exposures vary. Our earlier work [11] uses the method by HaCohen et al [9] to find dense correspondences; wrong correspondences are then detected and corrected by pasting small patches transformed with local homographies. The recent method by Sen et al [24] 1 converts each image into a linear space inverting the camera response function, and selects an image as the reference for the final HDR image.…”

“…Our contribution is a novel method for generating a registered stack from a set of mis-aligned images of dynamic scenes, similar to Hu et al [11] and Sen et al [24]. However, as opposed to their work, our algorithm can be applied to generic non-linearized exposure stacks, and is also capable of dealing with large saturated regions in the reference image, even under large camera motion or scene object displacements.…”

Probabilistic motion pixel detection for the reduction of ghost artifacts in high dynamic range images from multiple exposures

“…There are also patch-based methods, in which patches including moving objects are excluded [16,17]. To simultaneously deal with the misalignment and moving objects, Zimmer et al [18] proposed an optical flow-based energy minimization method, and Hu et al [19] used nonrigid dense correspondence and color transfer function for this task. Recently, low-rank matrix-based algorithms [20,21] have also been presented, based on the assumption that irradiance maps are linearly related to LDR exposures.…”