Learning to Reduce Defocus Blur by Realistically Modeling Dual-Pixel Data

Abuolaim, Abdullah; Delbracio, Mauricio; Kelly, Damien P.; Brown, Michael S.; Milanfar, Peyman

doi:10.1109/iccv48922.2021.00229

Cited by 44 publications

(25 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Geometry dataset for Dual-Pixel. Owing to the growing research interest in DP photography, several real/synthetic DP datasets [3,15,35,36] have been released. Garg et al [15] propose a real-world DP dataset that includes scenescale images captured by an array of smartphones.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Facial Depth and Normal Estimation using Single Dual-Pixel Camera

Kang¹,

Choe²,

Ha³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

Section: Related Workmentioning

confidence: 99%

“…3. These cameras are available to capture DP images [2,3,35]. Each camera is equipped with a Canon 135mm L lens.…”

Section: Hardware Setupmentioning

confidence: 99%

Facial Depth and Normal Estimation using Single Dual-Pixel Camera

Kang¹,

Choe²,

Ha³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…3 Experiments are conducted on the real-wold defocus deblurring DPDD dataset [23]. Both quantitative and qualitative results show that our DPANet is notably superior to the stateof-the-art methods including single image defocus deblurring methods [4], [7], [8], motion deblurring methods [27], [28] and DP defocus deblurring methods [22]- [24]. Especially on difficult cases, severe defocus blurry images can be well restored by our DPANet.…”

Section: Introductionmentioning

confidence: 97%

“…One image captured by DP sensors can be split into two-image views, whose stereo information is Y. Li, Y. Yi, D. Ren, Q. Li and W. Zuo are with the School of Computer Science and Technology, Harbin Institute of Technology, Harbin 150001, China (e-mail: liyuhit@outlook.com, csylyi@outlook.com, rendongweihit@gmail.com, qinceli@hit.edu.cn, cswmzuo@hit.edu.cn). beneficial to several computer vision tasks [16]- [21] as well as defocus deblurring [22]- [24]. In [23], Abuolaim et al established a real-world dual-pixel defocus deblurring (DPDD) dataset using a Cannon camera, based on which a basic encoder-decoder is trained to learn the mapping from DP views to the latent all-in-focus image.…”

Section: Introductionmentioning

confidence: 99%

“…In DDDNet [22], DP defocus deblurring and depth estimation are simultaneously modeled, in which depth estimation and defocus deblurring are suggested to benefit each other. In RDPD [24], a realistic DP defocus model and a recurrent convolution network are proposed to tackle DP defocus deblurring. Although these DP defocus deblurring methods have notably outperformed single image defocus deblurring methods [4]- [9], they take naive concatenation of DP views as input, while neglecting the disparity between left and right views.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning Dual-Pixel Alignment for Defocus Deblurring

Liu¹,

Yi²,

Ren³

et al. 2022

Preprint

View full text Add to dashboard Cite

It is a challenging task to recover all-in-focus image from a single defocus blurry image in real-world applications. On many modern cameras, dual-pixel (DP) sensors create two-image views, based on which stereo information can be exploited to benefit defocus deblurring. Despite existing DP defocus deblurring methods achieving impressive results, they directly take naive concatenation of DP views as input, while neglecting the disparity between left and right views in the regions out of camera's depth of field (DoF). In this work, we propose a Dual-Pixel Alignment Network (DPANet) for defocus deblurring. Generally, DPANet is an encoder-decoder with skip-connections, where two branches with shared parameters in the encoder are employed to extract and align deep features from left and right views, and one decoder is adopted to fuse aligned features for predicting the all-infocus image. Due to that DP views suffer from different blur amounts, it is not trivial to align left and right views. To this end, we propose novel encoder alignment module (EAM) and decoder alignment module (DAM). In particular, a correlation layer is suggested in EAM to measure the disparity between DP views, whose deep features can then be accordingly aligned using deformable convolutions. And DAM can further enhance the alignment of skip-connected features from encoder and deep features in decoder. By introducing several EAMs and DAMs, DP views in DPANet can be well aligned for better predicting latent all-in-focus image. Experimental results on real-world datasets show that our DPANet is notably superior to state-of-the-art deblurring methods in reducing defocus blur while recovering visually plausible sharp structures and textures.

show abstract