Learning to Generate Multi-Exposure Stacks With Cycle Consistency for High Dynamic Range Imaging

Lee, Siyeong; Jo, So Yeon; An, Gwon Hwan; Kang, Suk‐Ju

doi:10.1109/tmm.2020.3013378

Cited by 18 publications

(17 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, the potential of GT HDR images has not yet been fully exploited. Existing methods only leverage HDR images for supervision; however, they can be used to learn privileged knowledge, as in [160]. Therefore, it is possible to add a removable network that encodes the feature representations of HDR images to guide the learning of LDR image features.…”

Section: Discussion and New Perspectivesmentioning

confidence: 99%

See 1 more Smart Citation

Deep Learning for HDR Imaging: State-of-the-Art and Future Trends

Wang

Yoon

2021

Preprint

View full text Add to dashboard Cite

High dynamic range (HDR) imaging is a technique that allows an extensive dynamic range of exposures, which is important in image processing, computer graphics, and computer vision. In recent years, there has been a significant advancement in HDR imaging using deep learning (DL). This study conducts a comprehensive and insightful survey and analysis of recent developments in deep HDR imaging methodologies. We hierarchically and structurally group existing deep HDR imaging methods into five categories based on (1) number/domain of input exposures, (2) number of learning tasks, (3) novel sensor data, (4) novel learning strategies, and (5) applications. Importantly, we provide a constructive discussion on each category regarding its potential and challenges. Moreover, we review some crucial aspects of deep HDR imaging, such as datasets and evaluation metrics. Finally, we highlight some open problems and point out future research directions.

show abstract

Section: Discussion and New Perspectivesmentioning

confidence: 99%

“…[158] then adopted cGAN [159] to learn HR and high-quality tone mapping. Considering that the problem of single-exposure HDR imaging is ill-posed, [160] proposed generating multi-exposure stacks using cGAN. A cycle consistency loss [101] was also used to preserve the exposure level.…”

Section: Hdr Imaging With Ganmentioning

confidence: 99%

Deep Learning for HDR Imaging: State-of-the-Art and Future Trends

Wang

Yoon

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Endo et al [12] used U-Net [24] to generate HDR images with wide brightness ranges by bracketing images with various exposure values. Similarly, Lee et al generated HDR images using a chain network to create multiple exposure stacks [13] and a generative adversarial network (GAN) [14], [15]. Since MEF methods generally use more than three LDR images, they require extensive processing time, power consumption, and storage capacity.…”

Section: A Multiple Exposure Fusion Methodsmentioning

confidence: 99%

“…The proposed 𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀 is similar to those previously used in [15], [19], and [23], but the proposed method enables more accurate image segmentation by setting the 𝑡𝑡ℎ𝑟𝑟. The generated 𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀 is also non-linear, i.e., the square of the difference, producing smoother segmentation boundaries.…”

Section: A Pre-processing: Ldr Image Segmentationmentioning

confidence: 97%

HDR Image Reconstruction Using Segmented Image Learning

Lee¹,

Sunwoo

2021

IEEE Access

View full text Add to dashboard Cite

Converting a low dynamic range (LDR) image into a high dynamic range (HDR) image produces an image that closely depicts the real world without requiring expensive devices. Recent deep learning developments can produce highly realistic and sophisticated HDR images. This paper proposes a deep learning method to segment the bright and dark regions from an input LDR image and reconstruct the corresponding HDR image with similar dynamic ranges in the real world. The proposed multi-stage deep learning network brightens bright regions and darkens dark regions, and features with extended brightness range are combined to form the HDR image. Dividing the LDR image into the bright and dark regions effectively implements information on lost over-exposed and under-exposed areas, reconstructing a natural HDR image with color and appearance that is similar to reality. Experimental results confirm that the proposed method achieves an 8.52% higher HDR visual difference predictor (HDR-VDP) and a 41.2% higher log exposure range than current methods. Qualitative evaluation also verifies that the proposed method generates images that are close in quality to the ground truth. INDEX TERMSHigh dynamic range imaging, inverse tone mapping, deep learning, single exposure imageFIGURE 1. Luminance ranges for high dynamic range (HDR, green), low dynamic range (LDR, blue) and human perception (red). Night Moon Indoor Sunlight Luminace (cd/m 2 ) Human perception LDR image HDR image

show abstract

“…Conventional MEF methods often suffer from ghost artifacts due to moving object(s) while acquiring multiple LDR images with parallax. The second one is so-called inverse tone mapping (iTM) [8][9][10][11][12][13][14], which reconstructs an HDR image using only a single LDR image.…”

Section: Introductionmentioning

confidence: 99%

Cyclic Learning-Based Lightweight Network for Inverse Tone Mapping

Park

Song

2022

Electronics

View full text Add to dashboard Cite

Recent studies on inverse tone mapping (iTM) have moved toward indirect mapping, which generates a stack of low dynamic range (LDR) images with multiple exposure values (multi-EV stack) and then merges them. In order to generate multi-EV stack(s), several large-scale networks with more than 20 M parameters have been proposed, but their high dynamic range (HDR) reconstruction and multi-EV stack generation performance were not acceptable. Also, some previous methods using cycle consistency should even have trained additional networks that are not used for multi-EV stack generation, which results in large memory for training. Thus, this paper proposes novel cyclic learning based on cycle consistency to reduce the memory burden in training. In detail, we eliminated networks used only for training, so the proposed method enables efficient learning in terms of training-purpose memory. In addition, this paper presents a lightweight iTM network that dramatically reduces the network sizes of the existing networks. Actually, the proposed lightweight network requires only a small parameter size of 1/100 compared to the state-of-the-art (SOTA) method. The lightweight network contributes to the practical use of iTM. Therefore, the proposed method based on a lightweight network reliably generates a multi-EV stack. Experimental results show that the proposed method achieves quantitatively SOTA performance and is qualitatively comparable to conventional indirect iTM methods.

show abstract

Learning to Generate Multi-Exposure Stacks With Cycle Consistency for High Dynamic Range Imaging

Cited by 18 publications

References 25 publications

Deep Learning for HDR Imaging: State-of-the-Art and Future Trends

Deep Learning for HDR Imaging: State-of-the-Art and Future Trends

HDR Image Reconstruction Using Segmented Image Learning

Cyclic Learning-Based Lightweight Network for Inverse Tone Mapping

Contact Info

Product

Resources

About