HyperInverter: Improving StyleGAN Inversion via Hypernetwork

Abstract

Real-world image manipulation has achieved fantastic progress in recent years as a result of the exploration and utilization of GAN latent spaces. GAN inversion is the first step in this pipeline, which aims to map the real image to the latent code faithfully. Unfortunately, the majority of existing GAN inversion methods fail to meet at least one of the three requirements listed below: high reconstruction quality, editability, and fast inference. We present a novel two-phase strategy in this research that fits all requirements at the same time. In the first phase, we train an encoder to map the input image to StyleGAN2 W-space, which was proven to have excellent editability but lower reconstruction quality. In the second phase, we supplement the reconstruction ability in the initial phase by leveraging a series of hypernetworks to recover the missing information during inversion. These two steps complement each other to yield high reconstruction quality thanks to the hypernetwork branch and excellent editability due to the inversion done in the W-space. Our method is entirely encoder-based, resulting in extremely fast inference. Extensive experiments on two challenging datasets demonstrate the superiority of our method. 1

State‐of‐the‐Art in the Architecture, Methods and Applications of StyleGAN

“…While most generator tuning approaches improve the image inversion via a per-image optimization of the generator weights, such an approach is costly in terms of inference time. To reduce this inference overhead, Alaluf et al [ATM * 21] and Dinh et al [DTNH21] propose a hypernetwork-based encoder that learns how to modify the pre-trained generator weights to best reconstruct a given image. Such a learned approach results in high-fidelity reconstructions and edits, at a fraction of the time compared to optimization-based tuning approaches.…”