Rethinking Spatially-Adaptive Normalization

Tan, Zhentao; Chen, Dongdong; Chu, Qi; Chai, Menglei; Liao, Jing; He, Mingming; Yuan, Lu; Yu, Nenghai

doi:10.48550/arxiv.2004.02867

Cited by 6 publications

(7 citation statements)

References 27 publications

(69 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Normalization layers without track running statistics introduce extra computation cost, and we take this into account for our calculation of MACs during pruning. Moreover, for GauGAN, we use synchronized batch normalization as suggested by previous work [58,67], and remove the spectral norm [55] as we find it does not have much impact on the model performance.…”

Section: Acknowledgementmentioning

confidence: 99%

Teachers Do More Than Teach: Compressing Image-to-Image Models

Jin¹,

Ren²,

Woodford³

et al. 2021

Preprint

View full text Add to dashboard Cite

Generative Adversarial Networks (GANs) have achieved huge success in generating high-fidelity images, however, they suffer from low efficiency due to tremendous computational cost and bulky memory usage. Recent efforts on compression GANs show noticeable progress in obtaining smaller generators by sacrificing image quality or involving a time-consuming searching process. In this work, we aim to address these issues by introducing a teacher network that provides a search space in which efficient network architectures can be found, in addition to performing knowledge distillation. First, we revisit the search space of generative models, introducing an inception-based residual block into generators. Second, to achieve target computation cost, we propose a one-step pruning algorithm that searches a student architecture from the teacher model and substantially reduces searching cost. It requires no 1 sparsity regularization and its associated hyper-parameters, simplifying the training procedure. Finally, we propose to distill knowledge through maximizing feature similarity between teacher and student via an index named Global Kernel Alignment (GKA). Our compressed networks achieve similar or even better image fidelity (FID, mIoU) than the original models with much-reduced computational cost, e.g., MACs. Code will be released at https://github.com/snap-research/CAT.

show abstract

Section: Acknowledgementmentioning

confidence: 99%

Teachers Do More Than Teach: Compressing Image-to-Image Models

Jin¹,

Ren²,

Woodford³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…We compare INADE with several SOTA works, including quality-oriented (pix2pixHD [42], SPADE [30], CLADE [38,37] and SEAN [51]) and diversity-oriented (BicycleGAN [50], DSCGAN [44], and GroupDNet [52]) methods. For a fair comparison, we directly use the pretrained models provided by the authors when available, otherwise train the models by ourselves using the codes and settings provided by the authors.…”

Section: Quantitative and Qualitative Comparisonsmentioning

confidence: 99%

“…We adopt this step to facilitate supervised training and enable test-time reference-based style guidance. Inspired by [30,38,37], the proposed method is called INADE (INstance-Adaptive DEnormalization).…”

Section: Introductionmentioning

confidence: 99%

Diverse Semantic Image Synthesis via Probability Distribution Modeling

Tan¹,

Chai²,

Chen³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Semantic image synthesis, translating semantic layouts to photo-realistic images, is a one-to-many mapping problem. Though impressive progress has been recently made, diverse semantic synthesis that can efficiently produce semantic-level multimodal results, still remains a challenge. In this paper, we propose a novel diverse semantic image synthesis framework from the perspective of semantic class distributions, which naturally supports diverse generation at semantic or even instance level. We achieve this by modeling class-level conditional modulation parameters as continuous probability distributions instead of discrete values, and sampling per-instance modulation parameters through instance-adaptive stochastic sampling that is consistent across the network. Moreover, we propose prior noise remapping, through linear perturbation parameters encoded from paired references, to facilitate supervised training and exemplar-based instance style control at test time. Extensive experiments on multiple datasets show that our method can achieve superior diversity and comparable quality compared to state-of-the-art methods. Code will be available at https://github.com/tzt101/ INADE.git

show abstract

“…Other choices have also been proposed to remedy the issue. For isntance, CLADE [66] proposed to use class-adaptive modulation parameters instead. CC-FPSE [47] employed spatially-varying convolutional weights instead of the spatially-varying nor-malization layers.…”

Section: Related Workmentioning

confidence: 99%

USIS: Unsupervised Semantic Image Synthesis

Eskandar¹,

Abdelsamad²,

Armanious³

et al. 2021

Preprint

View full text Add to dashboard Cite

Semantic Image Synthesis (SIS) is a subclass of image-to-image translation where a photorealistic image is synthesized from a segmentation mask. SIS has mostly been addressed as a supervised problem. However, state-of-the-art methods depend on a huge amount of labeled data and cannot be applied in an unpaired setting. On the other hand, generic unpaired image-to-image translation frameworks underperform in comparison, because they color-code semantic layouts and feed them to traditional convolutional networks, which then learn correspondences in appearance instead of semantic content. In this initial work, we propose a new Unsupervised paradigm for Semantic Image Synthesis (USIS) as a first step towards closing the performance gap between paired and unpaired settings. Notably, the framework deploys a SPADE generator that learns to output images with visually separable semantic classes using a self-supervised segmentation loss. Furthermore, in order to match the color and texture distribution of real images without losing highfrequency information, we propose to use whole image wavelet-based discrimination. We test our methodology on 3 challenging datasets and demonstrate its ability to generate multimodal photorealistic images with an improved quality in the unpaired setting.

show abstract

Rethinking Spatially-Adaptive Normalization

Cited by 6 publications

References 27 publications

Teachers Do More Than Teach: Compressing Image-to-Image Models

Teachers Do More Than Teach: Compressing Image-to-Image Models

Diverse Semantic Image Synthesis via Probability Distribution Modeling

USIS: Unsupervised Semantic Image Synthesis

Contact Info

Product

Resources

About