Sharpening Out of Focus Images using High‐Frequency Transfer

Abstract

Figure 1: Owing to out-of-focus errors, (a) and (b) are focused differently and capture different moments. There is significant motion between the images (notice the arms), as can also be seen by flipping between them in the HTML. With the assisting image (a) and our high frequency blending techniques, our result (c) uses the additional information to reduce ringing and add details. Readers are encouraged to zoom into the PDF, didactic video, and supplementary HTML images and comparisons. AbstractFocus misses are common in image capture, such as when the camera or the subject moves rapidly in sports and macro photography. One option to sharpen focus-missed photographs is through single image deconvolution, but high-frequency data cannot be fully recovered; therefore, artifacts such as ringing and amplified noise become apparent. We propose a new method that uses assisting, similar but different, sharp image(s) provided by the user (such as multiple images of the same subject in different positions captured using a burst of photographs). Our first contribution is to theoretically analyze the errors in three sources of data-a slightly sharpened original input image that we call the target, single image deconvolution with an aggressive inverse filter, and warped assisting image(s) registered using optical flow. We show that these three sources have different error characteristics, depending on image location and frequency band (for example, aggressive deconvolution is more accurate in high-frequency regions like edges). Next, we describe a practical method to compute these errors, given we have no ground truth and cannot easily work in the Fourier domain. Finally, we select the best source of data for a given pixel and scale in the Laplacian pyramid. We accurately transfer high-frequency data to the input, while minimizing artifacts. We demonstrate sharpened results on out-of-focus images in macro, sports, portrait and wildlife photography.

Non-Parametric Blur Map Regression for Depth of Field Extension

“…To account for its spatiallyvarying nature, the estimation of defocus blur can generally be cast as a blur map estimation [1]- [4], for which the scale parameter of a priori known defocus point-spread-functions (PSF) model (disc, Gaussian) needs to be specified at each pixel. This has proven hard to solve accurately, and, to this date, most successful solutions for out-of-focus restoration are thus techniques for which the blur kernels estimation is either simplified or circumvented thanks to alterations in the optical design (coded aperture [5], chromatic aberrations [6]) or the presence of correctly focused images of the same scene [7]. This contrasts to camera shake deblurring, for which a broad range of methods [8]- [16] effectively work based on a single image recorded with a conventional camera.…”

Development of a Smartphone-Based Skin Simulation Model for Medical Education

“…Similarly, a soft, low-cost, pneumatic, or hydraulic-based elastomer approach could be an alternative for a reconfigurable, programmable elastomer 29,30 . Other image correction and enhancement methods can also be explored, including deconvolution-based approaches to correct out-of-focus images displayed over the smartphone 31 . In the future, we will incorporate this model in the IOS platform.…”