Image Restoration from Parametric Transformations using Generative Models

Abstract

When images are statistically described by a generative model we can use this information to develop optimum techniques for various image restoration problems as inpainting, super-resolution, image coloring, generative model inversion, etc. With the help of the generative model it is possible to formulate, in a natural way, these restoration problems as Statistical estimation problems. Our approach, by combining maximum a-posteriori probability with maximum likelihood estimation, is capable of restoring images that are distorted by transformations even when the latter contain unknown parameters. The resulting optimization is completely defined with no parameters requiring tuning. This must be compared with the current state of the art which requires exact knowledge of the transformations and contains regularizer terms with weights that must be properly defined. Finally, we must mention that we extend our method to accommodate mixtures of multiple images where each image is described by its own generative model and we are able of successfully separating each participating image from a single mixture.Preprint. Under review.

Model Adaptation for Inverse Problems in Imaging

“…In Figure 9 we demonstrate the effect of ablating the proximity term in (5), which we believe are necessary to avoid degenerate solutions. Optimizing (5) or (9) without proximity terms fails to leverage the network learned for A 0 using ground truth images x. Without the proximity term, our solution relies too heavily on the calibration data without ground truth information.…”

“…Recent work [2,5,6] has explored solving inverse problems using generative networks, which permit reconstruction under arbitrary forward models assuming an expressive enough generative network. In particular, [2] and [5] consider the case where the forward model is either partially or entirely unknown, and hence may be learned by parameterizing and jointly optimizing over both the forward model and the latent code for the generative network.…”

Image De-Quantization Using Generative Models as Priors

“…In this work we adopt the second category of constraints and assume that a generative model is available that captures the statistical behavior of the original image. We basically intend to specialize the results developed for image restoration [8] to the de-quantization problem. This specialization is not straightforward since it demands proper mathematical analysis and the definition of suitable functions which are not mentioned in the general problem treated in [8].…”

“…We basically intend to specialize the results developed for image restoration [8] to the de-quantization problem. This specialization is not straightforward since it demands proper mathematical analysis and the definition of suitable functions which are not mentioned in the general problem treated in [8]. As in [8], we intend to reach our goal through a rigorous mathematical analysis which is based on the classical statistical estimation theory and which will lead us to a very well defined optimization problem, the solution of which will provide the desired estimate.…”

“…This specialization is not straightforward since it demands proper mathematical analysis and the definition of suitable functions which are not mentioned in the general problem treated in [8]. As in [8], we intend to reach our goal through a rigorous mathematical analysis which is based on the classical statistical estimation theory and which will lead us to a very well defined optimization problem, the solution of which will provide the desired estimate. Regarding the final optimization we must add that it will not contain any parameters that require fine-tuning using proper pre-processing, as is the case with the existing methods we mentioned before.…”