HDR image reconstruction from a single exposure using deep CNNs

Eilertsen, Gabriel; Kronander, Joel; Dénes, György; Mantiuk, Rafał; Unger, Jonas

doi:10.1145/3130800.3130816

Cited by 501 publications

(559 citation statements)

References 56 publications

(79 reference statements)

Supporting

Mentioning

555

Contrasting

Unclassified

Order By: Relevance

“…Reverse tone mapping can also be used to correct exposure of an image by inferring a high dynamic range (HDR) image from a single low dynamic range (LDR) input [MG16,EKM17,EKD*17]. Our work differs them in two aspects.…”

Section: Related Workmentioning

confidence: 99%

Dual Illumination Estimation for Robust Exposure Correction

Zhang

Nie

Zheng

2019

Computer Graphics Forum

105

View full text Add to dashboard Cite

Exposure correction is one of the fundamental tasks in image processing and computational photography. While various methods have been proposed, they either fail to produce visually pleasing results, or only work well for limited types of image (e.g., underexposed images). In this paper, we present a novel automatic exposure correction method, which is able to robustly produce high‐quality results for images of various exposure conditions (e.g., underexposed, overexposed, and partially under‐ and over‐exposed). At the core of our approach is the proposed dual illumination estimation, where we separately cast the under‐and over‐exposure correction as trivial illumination estimation of the input image and the inverted input image. By performing dual illumination estimation, we obtain two intermediate exposure correction results for the input image, with one fixes the underexposed regions and the other one restores the overexposed regions. A multi‐exposure image fusion technique is then employed to adaptively blend the visually best exposed parts in the two intermediate exposure correction images and the input image into a globally well‐exposed image. Experiments on a number of challenging images demonstrate the effectiveness of the proposed approach and its superiority over the state‐of‐the‐art methods and popular automatic exposure correction tools.

show abstract

Section: Related Workmentioning

confidence: 99%

Dual Illumination Estimation for Robust Exposure Correction

Zhang

Nie

Zheng

2019

Computer Graphics Forum

105

View full text Add to dashboard Cite

show abstract

“…The first one is based on the substitution, inside the proposed method, of the PatchMatch algorithm with a state‐of‐the‐art ITMOs, to do the expansion of the renderings before their backprojection on the cloud. We tested two image ITMOs, the global operator by Landis et al [Lan02](LANDIS) and the expand‐map based operator by Banterle et al [BLDC06](B anterle ), and two recent deep learning networks proposed by Eilertsen et al [EKD∗ 17](E iler ) and by Endo et al [EKM17](E ndo ). For classical ITMOs, we focused on non‐linear operators because they typically provide high‐quality results for IBL [BDA∗ 09].…”

Section: Resultsmentioning

confidence: 99%

High Dynamic Range Point Clouds for Real‐Time Relighting

Sabbadin

Palma

Banterle

et al. 2019

Computer Graphics Forum

View full text Add to dashboard Cite

Acquired 3D point clouds make possible quick modeling of virtual scenes from the real world. With modern 3D capture pipelines, each point sample often comes with additional attributes such as normal vector and color response. Although rendering and processing such data has been extensively studied, little attention has been devoted using the light transport hidden in the recorded per‐sample color response to relight virtual objects in visual effects (VFX) look‐dev or augmented reality (AR) scenarios. Typically, standard relighting environment exploits global environment maps together with a collection of local light probes to reflect the light mood of the real scene on the virtual object. We propose instead a unified spatial approximation of the radiance and visibility relationships present in the scene, in the form of a colored point cloud. To do so, our method relies on two core components: High Dynamic Range (HDR) expansion and real‐time Point‐Based Global Illumination (PBGI). First, since an acquired color point cloud typically comes in Low Dynamic Range (LDR) format, we boost it using a single HDR photo exemplar of the captured scene that can cover part of it. We perform this expansion efficiently by first expanding the dynamic range of a set of renderings of the point cloud and then projecting these renderings on the original cloud. At this stage, we propagate the expansion to the regions not covered by the renderings or with low‐quality dynamic range by solving a Poisson system. Then, at rendering time, we use the resulting HDR point cloud to relight virtual objects, providing a diffuse model of the indirect illumination propagated by the environment. To do so, we design a PBGI algorithm that exploits the GPU's geometry shader stage as well as a new mipmapping operator, tailored for G‐buffers, to achieve real‐time performances. As a result, our method can effectively relight virtual objects exhibiting diffuse and glossy physically‐based materials in real time. Furthermore, it accounts for the spatial embedding of the object within the 3D environment. We evaluate our approach on manufactured scenes to assess the error introduced at every step from the perfect ground truth. We also report experiments with real captured data, covering a range of capture technologies, from active scanning to multiview stereo reconstruction.

show abstract

“…Auto‐encoder has been widely applied in many fields such as image denoising, image generation, and reverse tone mapping problem . In general, it first transforms the input image into a low‐dimensional representative vector via the encoder part and then reconstruct the output through the decoder part.…”

Section: Methodsmentioning

confidence: 99%

Cascaded network with deep intensity manipulation for scene understanding

Yang

Wang

Chen

et al. 2019

Computer Animation & Virtual

View full text Add to dashboard Cite

Scene understanding is essential to robotic navigation and autonomous driving as it provides semantic information to their controlling system. However, it will fail when processing low‐light images/videos captured under adverse weather or at night use state‐of‐the‐art scene understanding methods. A naive way to directly infer semantics from low‐light images is ill posed because the low‐light condition distorts pixel intensities and buries details. In order to address this problem, we propose the Deep Intensity Manipulation Network (DIMNet), which could relight the input images and recover the details, and combine the DIMNet with a scene understanding network to get a cascaded network to learn the semantics from low‐light images. Through learning pixel intensity manipulation, our method can generate images not only visually pleasing but also practical for scene understanding. Qualitative and quantitative experiments demonstrate that the proposed method is effective and robust for both synthetic and real‐world images.

show abstract

HDR image reconstruction from a single exposure using deep CNNs

Cited by 501 publications

References 56 publications

Dual Illumination Estimation for Robust Exposure Correction

Dual Illumination Estimation for Robust Exposure Correction

High Dynamic Range Point Clouds for Real‐Time Relighting

Cascaded network with deep intensity manipulation for scene understanding

Contact Info

Product

Resources

About