85 pages HDR (High Dynamic Range) images have traditionally been obtained by merging multiple exposures each captured with a different exposure time. However, this approach entails longer capture times and necessitates deghosting if the captured scene contains moving objects. With the advent of modern camera sensors that can perform per-pixel exposure modulation, it is now possible to capture all of the required exposures within a single shot. The new challenge then becomes how to best combine different pixels with different exposure values into a single full-resolution and low-noise HDR image. In this thesis, we propose a joint multi-exposure frame deinterlacing and denoising algorithm powered by deep convolutional neural networks (DCNN). In our algorithm, we first train two DCNNs, with one tuned for reconstructing low exposures and the other for high exposures. Each DCNN takes the same mosaicked dual-ISO input image and outputs either the low exposure or high exposure depending on the type of the network. The resulting exposures can be demosaicked and converted to the desired target color space prior to HDR assembly. Our evaluations using computational metrics as well as visual comparisons indicate that the quality of our reconstructions v significantly surpasses the state-of-the-art in this field.

No abstract

No abstract

Video frame interpolation (VFI) enables many important applications that might involve the temporal domain, such as slow motion playback, or the spatial domain, such as stop motion sequences. We are focusing on the former task, where one of the key challenges is handling high dynamic range (HDR) scenes in the presence of complex motion. To this end, we explore possible advantages of dual-exposure sensors that readily provide sharp short and blurry long exposures that are spatially registered and whose ends are temporally aligned. This way, motion blur registers temporally continuous information on the scene motion that, combined with the sharp reference, enables more precise motion sampling within a single camera shot. We demonstrate that this facilitates a more complex motion reconstruction in the VFI task, as well as HDR frame reconstruction that so far has been considered only for the originally captured frames, not in-between interpolated frames. We design a neural network trained in these tasks that clearly outperforms existing solutions. We also propose a metric for scene motion complexity that provides important insights into the performance of VFI methods at the test time. Supplementary videos can be found at: deephdr.mpi-inf.mpg.de/.