Towards Practical and Efficient High-Resolution HDR Deghosting with CNN

Prabhakar, K. Ram; Agrawal, Susmit; Singh, Durgesh Kumar; Ashwath, Balraj; Babu, R. Venkatesh

doi:10.1007/978-3-030-58589-1_30

Cited by 53 publications

(38 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…misalignments, overexposed regions, and focus instead on desirable details of non-reference frames that might be missing in the reference frame. In the work of Prabhakar et al [14] parts of the com-putation, including the optical flow estimation, are performed in a lower resolution and later upscaled back to full resolution using a guide image generated with a simple weight map, thus saving some computation.…”

Section: Related Workmentioning

confidence: 99%

FlexHDR: Modelling Alignment and Exposure Uncertainties for Flexible HDR Imaging

Catley-Chandar,

Tanay,

Vandroux

et al. 2022

Preprint

View full text Add to dashboard Cite

High dynamic range (HDR) imaging is of fundamental importance in modern digital photography pipelines and used to produce a high-quality photograph with well exposed regions despite varying illumination across the image. This is typically achieved by merging multiple low dynamic range (LDR) images taken at different exposures. However, over-exposed regions and misalignment errors due to poorly compensated motion result in artefacts such as ghosting. In this paper, we present a new HDR imaging technique that specifically models alignment and exposure uncertainties to produce high quality HDR results. We introduce a strategy that learns to jointly align and assess the alignment and exposure reliability using an HDR-aware, uncertainty-driven attention map that robustly merges the frames into a single high quality HDR image. Further, we introduce a progressive, multi-stage image fusion approach that can flexibly merge any number of LDR images in a permutation-invariant manner. Experimental results show our method can produce better quality HDR images with up to 0.8dB PSNR improvement to the state-of-the-art, and subjective improvements in terms of better detail, colours, and fewer artefacts.

show abstract

Section: Related Workmentioning

confidence: 99%

FlexHDR: Modelling Alignment and Exposure Uncertainties for Flexible HDR Imaging

Catley-Chandar,

Tanay,

Vandroux

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Prabhakar et al [20] aggregate features derived by shared CNN modules to create a scalable architecture that can fuse an arbitrary number of images without retraining. Prabhakar et al [22] propose a method to minimize the consumption of memory and enable the fusion of high-resolution images. They perform alignment on lowresolution inputs and upsample intermediate features while generating the final HDR using a Bilateral Grid Upsampler [61].…”

Section: Related Workmentioning

confidence: 99%

“…They perform alignment on lowresolution inputs and upsample intermediate features while generating the final HDR using a Bilateral Grid Upsampler [61]. In spite of their better performance, most of the existing deep-learning methods [18,19,21,22,59,60] are not scalable to fuse arbitrary number of images without re-training.…”

Section: Related Workmentioning

confidence: 99%

“…Once trained, they can generate highquality HDR images faster than most other non-deep methods. A potential drawback, however, is that most existing learningbased methods today are not scalable [18,19,21,22]. We refer to scalability as a trait of a method to fuse arbitrary length LDR image sequences during inference.…”

Section: Introductionmentioning

confidence: 99%

“…Data-driven methods use learning algorithms, primarily Convolutional Neural Networks (CNN), to learn fusion as well as deghosting [18]- [22]. Once trained, they can generate highquality HDR images faster than most other non-deep methods.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Self-Gated Memory Recurrent Network for Efficient Scalable HDR Deghosting

Kathirvel¹,

Agrawal²,

Radhakrishnan³

2021

IEEE Trans. Comput. Imaging

Self Cite

View full text Add to dashboard Cite

We propose a novel recurrent network-based HDR deghosting method for fusing arbitrary length dynamic sequences. The proposed method uses convolutional and recurrent architectures to generate visually pleasing, ghosting-free HDR images. We introduce a new recurrent cell architecture, namely Self-Gated Memory (SGM) cell, that outperforms the standard LSTM cell while containing fewer parameters and having faster running times. In the SGM cell, the information flow through a gate is controlled by multiplying the gate's output by a function of itself. Additionally, we use two SGM cells in a bidirectional setting to improve output quality. The proposed approach achieves state-of-the-art performance compared to existing HDR deghosting methods quantitatively across three publicly available datasets while simultaneously achieving scalability to fuse variable length input sequence without necessitating re-training. Through extensive ablations, we demonstrate the importance of individual components in our proposed approach. The code is available at https://val.cds.iisc.ac.in/ HDR/HDRRNN/index.html.

show abstract