Texture Reconstruction Guided by a High-Resolution Patch

Abstract

In this paper, we aim at super-resolving a low-resolution texture under the assumption that a high-resolution patch of the texture is available. To do so, we propose a variational method that combines two approaches that are texture synthesis and image reconstruction. The resulting objective function holds a nonconvex energy that involves a quadratic distance to the low-resolution image, a histogram-based distance to the high-resolution patch, and a nonlocal regularization that links the missing pixels with the patch pixels. As for the histogram-based measure, we use a sum of Wasserstein distances between the histograms of some linear transformations of the textures. The resulting optimization problem is efficiently solved with a primal-dual proximal method. Experiments show that our method leads to a significant improvement, both visually and numerically, with respect to the state-of-the-art algorithms for solving similar problems.

Wasserstein Loss for Image Synthesis and Restoration

“…An initial version of our framework was presented in the thesis manuscript [62]. A related work [20] has been done in parallel with an application to super-resolution. These previous works advocate the use of optimal transport distances (also known as Wasserstein distances) to account in a robust and simple way for discrepancies between discrete empirical distributions.…”

Texture synthesis guided by a low-resolution image

“…The resulting problem is solved using an extension of the nonlocal total variation model, where a set of connections is built between the missing high-resolution pixels and a set of pixels that lies in the sample. However, to interpolate the missing data, the authors in [16] design a new nonlocal graph that provides better connections between the missing pixels and the high-resolution pixels. Additionally, they introduce a histogram-based statistical prior modeled by a sum of Wasserstein distances between the histogram of some linear transformations of the texture.…”

“…The work in [17] exploits a patch-based nonlocal regularization and a Generalized Gaussian model of the texture gradients, whose parameters are learned on the high-resolution patch. Differently from [15] and [16], the authors in [17] use a prior on the spatial covariance of the synthesized image.…”

“…In this paper, we propose a variational method that combines a histogram-based statistical prior [16] and a constraint on the power spectral density [17]. The main novelty of the proposed approach w.r.t.…”

“…The main novelty of the proposed approach w.r.t. [16] lies in introducing a spectral constraint instead of the nonlocal regularization, and in considering the texture synthesis problem from free noise acquisitions with a high down-sampling factor. On the other hand, the originality of our technique w.r.t.…”