A Variational Framework for Exemplar-Based Image Inpainting

Abstract

Non-local methods for image denoising and inpainting have gained considerable attention in recent years. This is in part due to their superior performance in textured images, a known weakness of purely local methods. Local methods on the other hand have demonstrated to be very appropriate for the recovering of geometric structures such as image edges. The synthesis of both types of methods is a trend in current research. Variational analysis in particular is an appropriate tool for a unified treatment of local and nonlocal methods. In this work we propose a general variational framework for non-local image inpainting, from which important and representative previous inpainting schemes can be derived, in addition to leading to novel ones. We explicitly study some of these, relating them to previous work and showing results on synthetic and real images.

4. Rotation invariance in exemplar-based image inpainting

“…In this thesis we want to enrich the capability of exemplar-based inpainting methods presented in [AFC11,AFCS11] with an additional degree of freedom, by allowing the algorithm to rotate the available patches by arbitrary rotation angles. As a consequence the variety of available patches is heavily increased and thereby the potential of the algorithm to recreate geometry and structure enhanced.…”

“…3.3). A further original result is the extension of the analysis in [AFC11,AFCS11], where rotations were not incorporated. In this thesis, the analysis is developed very carefully and in detail for the case of arbitrary rotation angles.…”

“…In this thesis, the analysis is developed very carefully and in detail for the case of arbitrary rotation angles. In particular, following the tracks of the proof of the fundamental Lemma [AFC11, Lemma 1], we not only extend the proof to rotation invariance but also develop the details of the proof, which were not completely developed in [AFC11,AFCS11], revealing, for instance, some mildly imprecise statements in the original proof within the technical report [AFC11].…”

“…In Section 2.5 we present a novel approach for the comparison of the gradients of two patches in a weighted L 2 -norm, whenever the weight functions are suitably chosen. For the application to exemplar-based inpainting, this permits an efficient treatment of the so called patch non-local Poisson scheme, which was introduced in [AFCS11] without rotations, in the rotation invariant case. Our gradient-based patch comparison method has a similar computational cost as the weighted L 2 -norm approach from above.…”

“…The minimization of (1.2) with respect to Γ, whenever u is fixed coincides with the computation of the best matching patch in the L 2 -norm. This model has been subject of further analysis in [AFC11,AFCS11], where existence of minimizers and convergence of the alternated optimization scheme to critical points of the functional in (1.2) was proven. Furthermore, other error functions apart from the L 2 -norm have been introduced, which measure the distance of the patterns on two patches and also interfere with the image synthesis.…”