Focusing on Out-of-Focus: Assessing Defocus Estimation Algorithms for the Benefit of Automated Image Masking

Abstract

Acquiring photographs as input for an image-based modelling pipeline is less trivial than often assumed. Photographs should be correctly exposed, cover the subject sufficiently from all possible angles, have the required spatial resolution, be devoid of any motion blur, exhibit accurate focus and feature an adequate depth of field. The last four characteristics all determine the “sharpness” of an image and the photogrammetric, computer vision and hybrid photogrammetric computer vision communities all assume that the object to be modelled is depicted “acceptably” sharp throughout the whole image collection. Although none of these three fields has ever properly quantified “acceptably sharp”, it is more or less standard practice to mask those image portions that appear to be unsharp due to the limited depth of field around the plane of focus (whether this means blurry object parts or completely out-of-focus backgrounds). This paper will assess how well- or ill-suited defocus estimating algorithms are for automatically masking a series of photographs, since this could speed up modelling pipelines with many hundreds or thousands of photographs. To that end, the paper uses five different real-world datasets and compares the output of three state-of-the-art edge-based defocus estimators. Afterwards, critical comments and plans for the future finalise this paper.

Handling Critical Aspects in Massive Photogrammetric Digitization of Museum Assets

“…Effective but timeconsuming solutions, like focus stacking or shape from focus (Niederöst et al, 2003), can solve this well-known problem in macro photography, but they are hardly applicable in massive digitisation. Further processing approaches rely on masking unsharp and out-of-focus areas by also resorting to automatic defocus estimating algorithms (Verhoeven, 2018), which present several limitations. However, DoF effects in 3D reconstruction is still a topic scarcely investigated (Menna et al, 2012;Sapirstein, 2018;Lastilla et al, 2019;Webb et al, 2020).…”

“…Following Menna et al (2012), the CoC should not be set larger than the required resolution: it is the diameter of the blur spot measured on the sensor, calculated as the ratio GSD/S. Images must be "acceptably" sharp for the image-based pipeline, although the range of acceptability embodied in the circle of confusion has not been quantified yet (Verhoeven, 2018). In massive digitization, selecting adequate camera settings for avoiding or limiting unsharp areas can prevent further image pre-or post-processing efforts.…”

Quantifying Depth of Field and Sharpness for Image-Based 3d Reconstruction of Heritage Objects

“…Masking has been used to improve the quality of the alignment and decrease the reconstruction processing time, so by streamlining methods masking from sharpness would be beneficial. Future research could include testing with image stacking to improve the reliability of the camera calibration with the stacking workflow, investigating focus measure operators from Shape From Focus (SFF) as a means for masking based on sharpness, and even testing the Matlab toolbox presented by Verhoeven (2018)…”

“…DOF describes the range of acceptable image sharpness both in front of and behind the plane of sharp focus. While DOF can be quantified for photographic imaging, an "acceptable" value for the image sharpness has not yet been quantified in the photogrammetric and computer vision communities where images are to be used for 3D reconstruction (Verhoeven, 2018).…”

“…The visual and scientific value of output 3D reconstructions also benefit from low image noise and consistent textures for image draping. Whilst the need for consistent DOF across an imaged object is well recognized, the DOF requirement for 3D reconstruction given its definition from perceptual quantity and the idea of "acceptable" sharpness embodied in the circle of confusion has yet to be quantified (Verhoeven, 2018).…”