Defocus Map Estimation and Deblurring from a Single Dual-Pixel Image

Xin, Shumian; Wadhwa, Neal; Xue, Tao; Barron, Jonathan T.; Srinivasan, Pratul P.; Chen, Jiawen; Gkioulekas, Ioannis; Garg, Rahul

doi:10.1109/iccv48922.2021.00223

Cited by 26 publications

(7 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Better network designs have been proposed, such as iterative adaptive [9] and kernel-sharing parallel atrous [10] convolutions. With the help of dual-pixel images, multi-task learning methods such as defocus estimation [36] and predicting dualpixel views [37] are also proved to benefit defocus deblurring. Synthetic datasets such as dual-pixel video sequences [38] and depth images [39] have also been used as assistance for defocus deblurring.…”

Section: B Defocus Deblurringmentioning

confidence: 99%

Point-and-Shoot All-in-Focus Photo Synthesis From Smartphone Camera Pair

Luo

Peng

Zhao

et al. 2023

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

All-in-Focus (AIF) photography is expected to be a commercial selling point for modern smartphones. Standard AIF synthesis requires manual, time-consuming operations such as focal stack compositing, which is unfriendly to ordinary people. To achieve point-and-shoot AIF photography with a smartphone, we expect that an AIF photo can be generated from one shot of the scene, instead of from multiple photos captured by the same camera. Benefiting from the multi-camera module in modern smartphones, we introduce a new task of AIF synthesis from main (wide) and ultra-wide cameras. The goal is to recover sharp details from defocused regions in the main-camera photo with the help of the ultra-wide-camera one. The camera setting poses new challenges such as parallax-induced occlusions and inconsistent color between cameras. To overcome the challenges, we introduce a predict-and-refine network to mitigate occlusions and propose dynamic frequency-domain alignment for color correction. To enable effective training and evaluation, we also build an AIF dataset with 2686 unique scenes. Each scene includes two photos captured by the main camera, one photo captured by the ultrawide camera, and a synthesized AIF photo. Results show that our solution, termed EasyAIF, can produce high-quality AIF photos and outperforms strong baselines quantitatively and qualitatively. For the first time, we demonstrate point-and-shoot AIF photo synthesis successfully from main and ultra-wide cameras.

show abstract

Section: B Defocus Deblurringmentioning

confidence: 99%

Point-and-Shoot All-in-Focus Photo Synthesis From Smartphone Camera Pair

Luo

Peng

Zhao

et al. 2023

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

show abstract

“…(Lee et al 2019) proposed a large dataset to estimate densely defocus maps. (Xin et al 2021) estimated the defocus map using a calibrated BK in an unsupervised way. (Liang et al 2022) used the DP views to estimate the defocus map for the first time.…”

Section: Defocus Deblurringmentioning

confidence: 99%

“…Although DP views were initially used in autofocus tasks (Abuolaim, Punnappurath, and Brown 2018;Abuolaim and Brown 2020b), more applications have been discovered, including defocus deblurring (Vo 2021), depth estimation (Garg et al 2019;Punnappurath et al 2020;Wu et al 2021;Kang et al 2021), stereo matching (Zhang et al 2020;Pan et al 2021), reflection removal (Punnappurath and Brown 2019), synthetic DoF (Wadhwa et al 2018) and motion synthesis (Abuolaim, Afifi, and Brown 2022b). Subsequently, Recurrent Neural Networks (Abuolaim et al 2021a) and Multiplane Image (Xin et al 2021) were also used in DP defocus deblurring task. (Liang et al 2022) proposed BaMBNet, a blur-aware network, which achieved better results.…”

Section: Defocus Deblurringmentioning

confidence: 99%

See 1 more Smart Citation

Learnable Blur Kernel for Single-Image Defocus Deblurring in the Wild

Zhai

Zeng

et al. 2023

AAAI

View full text Add to dashboard Cite

Recent research showed that the dual-pixel sensor has made great progress in defocus map estimation and image defocus deblurring. However, extracting real-time dual-pixel views is troublesome and complex in algorithm deployment. Moreover, the deblurred image generated by the defocus deblurring network lacks high-frequency details, which is unsatisfactory in human perception. To overcome this issue, we propose a novel defocus deblurring method that uses the guidance of the defocus map to implement image deblurring. The proposed method consists of a learnable blur kernel to estimate the defocus map, which is an unsupervised method, and a single-image defocus deblurring generative adversarial network (DefocusGAN) for the first time. The proposed network can learn the deblurring of different regions and recover realistic details. We propose a defocus adversarial loss to guide this training process. Competitive experimental results confirm that with a learnable blur kernel, the generated defocus map can achieve results comparable to supervised methods. In the single-image defocus deblurring task, the proposed method achieves state-of-the-art results, especially significant improvements in perceptual quality, where PSNR reaches 25.56 dB and LPIPS reaches 0.111.

show abstract

“…Defocus blur estimation has been actively studied in the literature. Some estimation methods used multiple images [15,16] or special hardware [17][18][19][20] while others methods rely on a single image. In this paper, we focus on single image defocus blur estimation methods.…”

Section: Introductionmentioning

confidence: 99%

Single image defocus map estimation through patch blurriness classification and its applications

Galetto

Deng

2022

Vis Comput

View full text Add to dashboard Cite

Depth information is useful in many image processing applications. However, since taking a picture is a process of projection of a 3D scene onto a 2D imaging sensor, the depth information is embedded in the image. Extracting the depth information from the image is a challenging task. A guiding principle is that the level of blurriness due to defocus is related to the distance between the object and the focal plane. Based on this principle and the widely used assumption that Gaussian blur is a good model for defocus blur, we formulate the problem of estimating the spatially varying defocus blurriness as a Gaussian blur classification problem. We solve the problem by training a deep neural network to classify image patches into one of the 20 levels of blurriness. We have created a dataset of more than 500,000 image patches of size $$32\times 32$$ 32 × 32 which does not require human labelling. The dataset is used to train and test several well-known network models. We find that MobileNetV2 is suitable for this application due to its low memory requirement and high accuracy. The trained model is used to determine the patch blurriness which is then refined by applying an iterative weighted guided filter. The result is a defocus map that carries the information of the degree of blurriness for each pixel. We compare the proposed method with state-of-the-art techniques and we demonstrate its successful applications in adaptive image enhancement and defocus magnification limited to images that present a clear distinction between defocus levels.

show abstract

Defocus Map Estimation and Deblurring from a Single Dual-Pixel Image

Cited by 26 publications

References 60 publications

Point-and-Shoot All-in-Focus Photo Synthesis From Smartphone Camera Pair

Point-and-Shoot All-in-Focus Photo Synthesis From Smartphone Camera Pair

Learnable Blur Kernel for Single-Image Defocus Deblurring in the Wild

Single image defocus map estimation through patch blurriness classification and its applications

Contact Info

Product

Resources

About