Regularized Training of Intermediate Layers for Generative Models for Inverse Problems

Abstract

Generative Adversarial Networks (GANs) have been shown to be powerful and flexible priors when solving inverse problems. One challenge of using them is overcoming representation error, the fundamental limitation of the network in representing any particular signal. Recently, multiple proposed inversion algorithms reduce representation error by optimizing over intermediate layer representations. These methods are typically applied to generative models that were trained agnostic of the downstream inversion algorithm. In our work, we introduce a principle that if a generative model is intended for inversion using an algorithm based on optimization of intermediate layers, it should be trained in a way that regularizes those intermediate layers. We instantiate this principle for two notable recent inversion algorithms: Intermediate Layer Optimization and the Multi-Code GAN prior. For both of these inversion algorithms, we introduce a new regularized GAN training algorithm and demonstrate that the learned generative model results in lower reconstruction errors across a wide range of under sampling ratios when solving compressed sensing, inpainting, and super-resolution problems.

Theoretical Perspectives on Deep Learning Methods in Inverse Problems

“…With a similar motivation but a very different approach, various methods were proposed in [63], [32], [50], [31] based on optimizing intermediate layers in the neural network defining G, which helps to expand the range of the generator and mitigate representation error. Conditions were given under which the required number of measurements is provably smaller than Theorem 2, and improvements in the out-ofdistribution robustness were observed experimentally.…”

Theoretical Perspectives on Deep Learning Methods in Inverse Problems

“…With a similar motivation but a very different approach, various methods were proposed in [59], [28], [46] based on optimizing intermediate layers in the neural network defining G, which helps to expand the range of the generator and mitigate representation error. Conditions were given under which the required number of measurements is provably smaller than Theorem 2, and improvements in the out-ofdistribution robustness were observed experimentally.…”