Transform Domain Pyramidal Dilated Convolution Networks for Restoration of Under Display Camera Images

Sethumadhavan, Hrishikesh Panikkasseril; Puthussery, Densen; Kuriakose, Melvin; Victor, Jiji Charangatt

doi:10.1007/978-3-030-68238-5_28

Cited by 11 publications

(9 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The results show that our method achieves state-of-the-art results for both the P-OLED track and T-OLED track. In particular, our DWFormer achieves 0.22 and 0.12 dB PSNR improvements over the previous best methods PDCRN [7] and DRANet [9] on the P-OLED and T-OLED tracks, respectively. Using both generated and real data to train our model, DWFormer can improve 1.01 dB on the P-OLED track and 0.59 dB on the T-OLED track over the previous one.…”

Section: Comparison Of Restoration Modelsmentioning

confidence: 86%

See 1 more Smart Citation

Image Restoration for Under-Display Camera

Zhou¹,

Ren

Emerton

et al. 2021

2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Under-display camera (UDC) provides an elegant solution for full-screen smartphones. However, UDC captured images suffer from severe degradation since sensors lie under the display. Although this issue can be tackled by image restoration networks, these networks require large-scale image pairs for training. To this end, we propose a modular network dubbed MPGNet trained using the generative adversarial network (GAN) framework for simulating UDC imaging. Specifically, we note that the UDC imaging degradation process contains brightness attenuation, blurring, and noise corruption. Thus we model each degradation with a characteristic-related modular network, and all modular networks are cascaded to form the generator. Together with a pixel-wise discriminator and supervised loss, we can train the generator to simulate the UDC imaging degradation process. Furthermore, we present a Transformer-style network named DWFormer for UDC image restoration. For practical purposes, we use depth-wise convolution instead of the multi-head self-attention to aggregate local spatial information. Moreover, we propose a novel channel attention module to aggregate global information, which is critical for brightness recovery. We conduct evaluations on the UDC benchmark, and our method surpasses the previous state-of-the-art models by 1.23 dB on the P-OLED track and 0.71 dB on the T-OLED track, respectively. Code is available at Github.

show abstract

Section: Comparison Of Restoration Modelsmentioning

confidence: 86%

“…In recent years, many learning-based methods [7,8,9,10] have been introduced to improve the quality of UDC images and made significant advancements as they can learn strong priors from large-scale datasets. However, such methods require large amounts of data, and collecting aligned image pairs is labor-intensive.…”

Section: Introductionmentioning

confidence: 99%

Image Restoration for Under-Display Camera

Zhou¹,

Ren

Emerton

et al. 2021

2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

show abstract

“…At ECCV 2020, Zhou et al for the first time held a UDC restoration challenge, resulting in four contributions [37,33,27,29]. The dataset for the challenge is captured on machine vision camera with a simulated UDC hardware.…”

Section: Isp-dependent and Monitor Data Udc Restorationmentioning

confidence: 99%

“…As can be seen, the small characters in the poster of the second column are a little blurry -even a little more blurry than the input image. All the papers based on ECCV 2020 UDC challenge [27,29,11,38] are all only trained in the monitor-captured dataset, which we think may have similar problems. The reason may cause this and more detail about this is explained in the supplement.…”

Section: Ablation Studymentioning

confidence: 99%

“…The Table 3 shows the result of monitor pretrained model before and after finetune. The PDCRN is the method [27] in the UDC chal-lenge. We can see that both our method and PDCRN has the same problem that although monitor pretrained model can perform very well in the monitor dataset, the metrics number decrease a lot when it test in the real world dataset.…”

Section: S32 Pretrain Dataset and Fine-tunementioning

confidence: 99%

See 1 more Smart Citation

ISP-Agnostic Image Reconstruction for Under-Display Cameras

Qi¹,

Li²,

Heidrich³

2021

Preprint

View full text Add to dashboard Cite

Under-display cameras have been proposed in recent years as a way to reduce the form factor of mobile devices while maximizing the screen area. Unfortunately, placing the camera behind the screen results in significant image distortions, including loss of contrast, blur, noise, color shift, scattering artifacts, and reduced light sensitivity.In this paper, we propose an image-restoration pipeline that is ISP-agnostic, i.e. it can be combined with any legacy ISP to produce a final image that matches the appearance of regular cameras using the same ISP. This is achieved with a deep learning approach that performs a RAW-to-RAW image restoration.To obtain large quantities of real under-display camera training data with sufficient contrast and scene diversity, we furthermore develop a data capture method utilizing an HDR monitor, as well as a data augmentation method to generate suitable HDR content. The monitor data is supplemented with real-world data that has less scene diversity but allows us to achieve fine detail recovery without being limited by the monitor resolution. Together, this approach successfully restores color and contrast as well as image detail.

show abstract

MIPI 2022 Challenge on Under-Display Camera Image Restoration: Methods and Results

Ran

Zhou

et al. 2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Transform Domain Pyramidal Dilated Convolution Networks for Restoration of Under Display Camera Images

Cited by 11 publications

References 16 publications

Image Restoration for Under-Display Camera

Image Restoration for Under-Display Camera

ISP-Agnostic Image Reconstruction for Under-Display Cameras

MIPI 2022 Challenge on Under-Display Camera Image Restoration: Methods and Results

Contact Info

Product

Resources

About