Solving Linear Inverse Problems Using Gan Priors: An Algorithm with Provable Guarantees

Shah, Viraj; Hegde, Chinmay

doi:10.1109/icassp.2018.8462233

Cited by 119 publications

(175 citation statements)

References 34 publications

Supporting

Mentioning

173

Contrasting

Order By: Relevance

“…A variety of follow-up works of [14] provided additional theoretical guarantees for compressive sensing with generative models. For example, instead of using a gradient descent algorithm as in [14], the authors of [16], [17] provide recovery guarantees for a projective gradient descent algorithm under various assumptions, where now the gradient steps are taken in the ambient space (i.e., on output vectors from the generative model)…”

Section: A Related Workmentioning

confidence: 99%

“…In practice, such a minimization problem may be hard, so it was proposed to use gradient descent in the latent space (i.e., on input vectors to the generative model).A variety of follow-up works of [14] provided additional theoretical guarantees for compressive sensing with generative models. For example, instead of using a gradient descent algorithm as in [14], the authors of [16], [17] provide recovery guarantees for a projective gradient descent algorithm under various assumptions, where now the gradient steps are taken in the ambient space (i.e., on output vectors from the generative model)In [14], the recovered signal is assumed to lie in (or close to) the range of the generative model G(·), which poses limitations for cases that the true signal is further from the range of G(·). To overcome this problem, a more general model allowing sparse deviations from the range of G(·) was introduced and analyzed in [18].In the case that the generative model is a ReLU network, under certain assumptions on the layer sizes and randomness assumptions on the weights, the authors of [19] show that the non-convex objective function given by empirical risk minimization does not have any spurious stationary points, and accordingly establish a simple gradient-based algorithm that is guaranteed to find the global minimum.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Information-Theoretic Lower Bounds for Compressive Sensing With Generative Models

Liu

Scarlett

2020

IEEE J. Sel. Areas Inf. Theory

View full text Add to dashboard Cite

The goal of standard compressive sensing is to estimate an unknown vector from linear measurements under the assumption of sparsity in some basis. Recently, it has been shown that significantly fewer measurements may be required if the sparsity assumption is replaced by the assumption that the unknown vector lies near the range of a suitably-chosen generative model. In particular, in (Bora et al., 2017) it was shown that roughly O(k log L) random Gaussian measurements suffice for accurate recovery when the k-input generative model is bounded and L-Lipschitz, and that O(kd log w) measurements suffice for k-input ReLU networks with depth d and width w. In this paper, we establish corresponding algorithm-independent lower bounds on the sample complexity using tools from minimax statistical analysis. In accordance with the above upper bounds, our results are summarized as follows: (i) We construct an L-Lipschitz generative model capable of generating group-sparse signals, and show that the resulting necessary number of measurements is Ω(k log L); (ii) Using similar ideas, we construct two-layer ReLU networks of high width requiring Ω(k log w) measurements, as well as lower-width deep ReLU networks requiring Ω(kd) measurements. As a result, we establish that the scaling laws derived in (Bora et al., 2017) are optimal or near-optimal in the absence of further assumptions. I. INTRODUCTIONOver the past 1-2 decades, tremendous research effort has been placed on theoretical and algorithmic studies of high-dimensional linear inverse problems [1], [2]. The prevailing approach has been to model low-dimensional structure via assumptions such as sparsity or low rankness, and numerous algorithmic approaches have been shown to be successful, including convex relaxations [3], [4], greedy methods [5], [6], and more. The problem of sparse estimation via linear measurements (commonly referred to as compressive sensing) is particularly well-understood, with theoretical developments including sharp performance bounds for both practical algorithms [4], [6]-[8] and (potentially intractable) information-theoretically optimal algorithms [9]-[12]. Following the tremendous success of deep generative models in a variety of applications [13], a new perspective on compressive sensing was recently introduced, in which the sparsity assumption is replaced by the assumption of the underlying signal being well-modeled by a generative model (typically corresponding to a deep neural network) [14]. This approach was seen to exhibit impressive performance in experiments, with reductions in the number of measurements by large factors such as 5 to 10 compared to sparsity-based methods. 2 In addition, [14] provided theoretical guarantees on their proposed algorithm, essentially showing that an L-Lipschitz generative model with bounded k-dimensional inputs leads to reliable recovery with m = O(k log L)random Gaussian measurements (see Section II for a precise statement). Moreover, for a ReLU network generative model from R k to R n with width w and depth d...

show abstract

Section: A Related Workmentioning

confidence: 99%

mentioning

confidence: 99%

Information-Theoretic Lower Bounds for Compressive Sensing With Generative Models

Liu

Scarlett

2020

IEEE J. Sel. Areas Inf. Theory

View full text Add to dashboard Cite

show abstract

“…We fixed the learning rate for projected gradient descent at 5e − 3. Note that, PGD has been successfully used in several applications involving GANs [8,13,52,57]. The optimal projection is given by G(z * ), where z * = arg min Here, each image is from a different run with the specified corruption setting, and we see that MimicGAN provides significantly better reprojection error in all cases above.…”

Section: Experiments Setupmentioning

confidence: 75%

MimicGAN: Robust Projection onto Image Manifolds with Corruption Mimicking

2020

View full text Add to dashboard Cite

In the past few years, Generative Adversarial Networks (GANs) have dramatically advanced our ability to represent and parameterize high-dimensional, non-linear image manifolds. As a result, they have been widely adopted across a variety of applications, ranging from challenging inverse problems like image completion, to problems such as anomaly detection and adversarial defense. A recurring theme in many of these applications is the notion of projecting an image observation onto the manifold that is inferred by the generator. In this context, Projected Gradient Descent (PGD) has been the most popular approach, which essentially optimizes for a latent vector that minimizes the discrepancy between a generated image and the given observation. However, PGD is a brittle optimization technique that fails to identify the right projection (or latent vector) when the observation is corrupted, or perturbed even by a small amount. Such corruptions are common in the real world, for example images in the wild come with unknown crops, rotations, missing pixels, or other kinds of non-linear distributional shifts which break current encoding methods, rendering downstream applications unusable. To address this, we propose corruption mimicking -a new robust projection technique, that utilizes a surrogate network to approximate the unknown corruption directly at test time, without the need for additional supervision or data augmentation. The proposed method is significantly more robust than PGD and other competing methods under a wide variety of corruptions, thereby enabling a more effective use of GANs in real-world applications. More importantly, we show that our approach produces state-of-the-art performance in several GAN-based applications -anomaly detection, domain adaptation, and adversarial defense, that benefit from an accurate projection.

show abstract

“…More specifically, this process involves sampling from the latent space of the GAN, in order to find an image that resembles the seed image. Though this has been conventionally carried out using projected gradient descent (PGD) [5,6], as we demonstrate in our results, this approach performs poorly when the initial estimate is too noisy or has too many artifacts, which is common under extremely limited angle scenarios.…”

Section: Introductionmentioning

confidence: 88%

“…and finally, X * = G(z * ) and can be solved using stochastic gradient descent. This has been referred to as a GAN prior [6] or a manifold prior [3] for inverse imaging. However, solving the equation of the form in (1) is not always possible since one may not have access to the measurement model A.…”

Section: Proposed Approachmentioning

confidence: 99%

Improving Limited Angle CT Reconstruction with a Robust GAN Prior

Kim

Thiagarajan

Mohan

et al. 2019

View full text Add to dashboard Cite

Limited angle CT reconstruction is an under-determined linear inverse problem that requires appropriate regularization techniques to be solved. In this work we study how pre-trained generative adversarial networks (GANs) can be used to clean noisy, highly artifact laden reconstructions from conventional techniques, by effectively projecting onto the inferred image manifold. In particular, we use a robust version of the popularly used GAN prior for inverse problems, based on a recent technique called corruption mimicking, that significantly improves the reconstruction quality. The proposed approach operates in the image space directly, as a result of which it does not need to be trained or require access to the measurement model, is scanner agnostic, and can work over a wide range of sensing scenarios.

show abstract

Solving Linear Inverse Problems Using Gan Priors: An Algorithm with Provable Guarantees

Cited by 119 publications

References 34 publications

Information-Theoretic Lower Bounds for Compressive Sensing With Generative Models

Information-Theoretic Lower Bounds for Compressive Sensing With Generative Models

MimicGAN: Robust Projection onto Image Manifolds with Corruption Mimicking

Improving Limited Angle CT Reconstruction with a Robust GAN Prior

Contact Info

Product

Resources

About