Bokeh Rendering from Defocus Estimation

Luo, Xianrui; Peng, Jiangde; Xian, Ke; Wu, Zijin; Cao, Zhiguo

doi:10.1007/978-3-030-67070-2_15

Cited by 12 publications

(5 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition to direct AIF synthesis by image fusion, defocus deblurring is also feasible to restore a sharp image from a bokeh image [27], [28]. Traditional methods apply a two-stage technique [29], [30], [31], [32], where defocus estimation [30], [32], [33], [34], [35] is first executed on a pre-defined blur model, then the predicted defocus map helps to deblur the image by non-blind deconvolution [29], [31].…”

Section: B Defocus Deblurringmentioning

confidence: 99%

Point-and-Shoot All-in-Focus Photo Synthesis From Smartphone Camera Pair

Luo

Peng

Zhao

et al. 2023

IEEE Trans. Circuits Syst. Video Technol.

Self Cite

View full text Add to dashboard Cite

All-in-Focus (AIF) photography is expected to be a commercial selling point for modern smartphones. Standard AIF synthesis requires manual, time-consuming operations such as focal stack compositing, which is unfriendly to ordinary people. To achieve point-and-shoot AIF photography with a smartphone, we expect that an AIF photo can be generated from one shot of the scene, instead of from multiple photos captured by the same camera. Benefiting from the multi-camera module in modern smartphones, we introduce a new task of AIF synthesis from main (wide) and ultra-wide cameras. The goal is to recover sharp details from defocused regions in the main-camera photo with the help of the ultra-wide-camera one. The camera setting poses new challenges such as parallax-induced occlusions and inconsistent color between cameras. To overcome the challenges, we introduce a predict-and-refine network to mitigate occlusions and propose dynamic frequency-domain alignment for color correction. To enable effective training and evaluation, we also build an AIF dataset with 2686 unique scenes. Each scene includes two photos captured by the main camera, one photo captured by the ultrawide camera, and a synthesized AIF photo. Results show that our solution, termed EasyAIF, can produce high-quality AIF photos and outperforms strong baselines quantitatively and qualitatively. For the first time, we demonstrate point-and-shoot AIF photo synthesis successfully from main and ultra-wide cameras.

show abstract

Section: B Defocus Deblurringmentioning

confidence: 99%

Point-and-Shoot All-in-Focus Photo Synthesis From Smartphone Camera Pair

Luo

Peng

Zhao

et al. 2023

IEEE Trans. Circuits Syst. Video Technol.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Deep Learning Method. With the development of deep learning, more attention has been paid to its use for automatic bokeh rendering [39,40,37,27]. Ignatov et al [19] propose an efficient multi-level CNN approach to render bokeh effects on a mobile GPU, named PyNET.…”

Section: Related Workmentioning

confidence: 99%

“…With the development of computer vision, depth estimation [14,30,34], defocus estimation [2,32,7] and saliency detection [54,38,43,56] have made significant progress, which also provide more directions for solving bokeh rendering tasks. Many methods [48,37,11,17] utilize these prior knowledge to synthesize bokeh effects. Luo et al [37] utilize radiance estimation and defocus estimation [47] to predict the relationship between the amount of blur and the intensity of pixels in each image, and then synthesizes the bokeh effect with pre-defined blur kernels.…”

Section: Related Workmentioning

confidence: 99%

“…Many methods [48,37,11,17] utilize these prior knowledge to synthesize bokeh effects. Luo et al [37] utilize radiance estimation and defocus estimation [47] to predict the relationship between the amount of blur and the intensity of pixels in each image, and then synthesizes the bokeh effect with pre-defined blur kernels. Depth priors seem to be more suitable for bokeh rendering tasks.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Constrained Predictive Filters for Single Image Bokeh Rendering

Zheng

Chen

Yuan

et al. 2022

IEEE Trans. Comput. Imaging

View full text Add to dashboard Cite

Bokeh rendering is a technique used to take pictures with out-of-focus areas to highlight regions of interest. Due to limitations in hardware and shooting condition, rendering a bokeh image from a full-focus image has attracted a lot of interest. In this paper, we model bokeh rendering as the combination of salient region retention and bokeh blurring, and propose a neural network to generate a realistic bokeh image from a single full-focus image through end-to-end training. Specifically, we propose a gate fusion block to estimate the salient area, and introduce a constrained predictive filter for salient region retention and bokeh blurring within a unified architecture. Further, we utilize a pixel coordinate-based map to enhance the training. Experimental results illustrate the effectiveness of our model. The comparison with state-of-the-art methods (PyNET [18], DMSHN [12], BGGAN [40], etc.) shows that our model produces better bokeh effects and retains salient objects.

show abstract

“…The solution of AIA-Smart team is based on defocus map estimation [28]. The proposed architecture consists of 4 modules (Fig.…”

Section: Aia-smartmentioning

confidence: 99%

AIM 2020 Challenge on Rendering Realistic Bokeh

Ignatov¹,

Timofte²,

Ming³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

This paper reviews the second AIM realistic bokeh effect rendering challenge and provides the description of the proposed solutions and results. The participating teams were solving a real-world bokeh simulation problem, where the goal was to learn a realistic shallow focus technique using a large-scale EBB! bokeh dataset consisting of 5K shallow / wide depth-of-field image pairs captured using the Canon 7D DSLR camera. The participants had to render bokeh effect based on only one single frame without any additional data from other cameras or sensors. The target metric used in this challenge combined the runtime and the perceptual quality of the solutions measured in the user study. To ensure the efficiency of the submitted models, we measured their runtime on standard desktop CPUs as well as were running the models on smartphone GPUs. The proposed solutions significantly improved the baseline results, defining the state-of-the-art for practical bokeh effect rendering problem. * A. Ignatov and R. Timofte ({andrey,radu.timofte}@vision.ee.ethz.ch, ETH Zurich) are the challenge organizers, while the other authors participated in the challenge. The Appendix A contains the authors' teams and affiliations. AIM 2020 webpage: https://data.vision.ee.ethz.ch/cvl/aim20/

show abstract

Bokeh Rendering from Defocus Estimation

Cited by 12 publications

References 38 publications

Point-and-Shoot All-in-Focus Photo Synthesis From Smartphone Camera Pair

Point-and-Shoot All-in-Focus Photo Synthesis From Smartphone Camera Pair

Constrained Predictive Filters for Single Image Bokeh Rendering

AIM 2020 Challenge on Rendering Realistic Bokeh

Contact Info

Product

Resources

About