GAN Compression: Efficient Architectures for Interactive Conditional GANs

Li, Muyang; Lin, Ji; Ding, Yaoyao; Liu, Zhijian; Zhu, Jun‐Yan; Han, Song

doi:10.1109/tpami.2021.3126742

Cited by 44 publications

(84 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Dynamic-OFA is a general approach for building dynamic DNNs, and the backbone network could be any super-networks trained by the OFA training pipeline. Our future work will investigate other applications such as network for IoT devices [12], transformers for natural language processing (NLP) tasks [18], generative adversarial networks (GANs) [11], 3D DNNs [17], etc.…”

Section: Discussionmentioning

confidence: 99%

Dynamic-OFA: Runtime DNN Architecture Switching for Performance Scaling on Heterogeneous Embedded Platforms

Lou¹,

Lei²,

Sabet³

et al. 2021

Preprint

View full text Add to dashboard Cite

Mobile and embedded platforms are increasingly required to efficiently execute computationally demanding DNNs across heterogeneous processing elements. At runtime, the available hardware resources to DNNs can vary considerably due to other concurrently running applications. The performance requirements of the applications could also change under different scenarios. To achieve the desired performance, dynamic DNNs have been proposed in which the number of channels/layers can be scaled in real time to meet different requirements under varying resource constraints. However, the training process of such dynamic DNNs can be costly, since platform-aware models of different deployment scenarios must be retrained to become dynamic. This paper proposes Dynamic-OFA, a novel dynamic DNN approach for state-of-the-art platform-aware NAS models (i.e. Once-for-all network (OFA)). Dynamic-OFA pre-samples a family of sub-networks from a static OFA backbone model, and contains a runtime manager to choose different sub-networks under different runtime environments. As such, Dynamic-OFA does not need the traditional dynamic DNN training pipeline. Compared to the state-of-the-art, our experimental results using ImageNet on a Jetson Xavier NX show that the approach is up to 3.5x (CPU), 2.4x (GPU) faster for similar ImageNet Top-1 accuracy, or 3.8% (CPU), 5.1% (GPU) higher accuracy at similar latency.

show abstract

Section: Discussionmentioning

confidence: 99%

Dynamic-OFA: Runtime DNN Architecture Switching for Performance Scaling on Heterogeneous Embedded Platforms

Lou¹,

Lei²,

Sabet³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…These techniques have seen applications in commercial image editing tools. However, due to their massive computation complexity and bulky size, applying generative models at scale is less practical, especially on resource-constrained platforms, where low memory foot-* Work done while at Snap Inc. [2,20,36,64,70] on Cy-cleGAN [85] for Horse Zebra dataset. Smaller MACs indicates more efficient models.…”

Section: Introductionmentioning

confidence: 99%

“…Our method (red star) achieves the state-of-the-art performance-efficiency tradeoff as it has the lowest FID with the smallest MACs. print, power consumption, and real-time execution are as, and often more, important than performance [36].…”

Section: Introductionmentioning

confidence: 99%

“…The problem of compressing generative models, on the other hand, is less investigated, despite that typical generators are bulky in memory usage and inefficient during inference. Up till now, only a handful of attempts exist [20,36,64,70], all of which degenerate the quality of synthetic images compared to the original model (Fig. 1).…”

Section: Introductionmentioning

confidence: 99%

“…In this work, we focus on compressing image-to-image translation networks, such as CycleGAN [85] and Gau-GAN [58]. Existing compression method [36] obtains an efficient student model and employs two additional networks: teacher and supernet, where the former is for knowledge distillation and the latter for architecture search. However, we argue that the supernet is not necessary, as the teacher can play its role.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Teachers Do More Than Teach: Compressing Image-to-Image Models

Jin¹,

Ren²,

Woodford³

et al. 2021

Preprint

View full text Add to dashboard Cite

Generative Adversarial Networks (GANs) have achieved huge success in generating high-fidelity images, however, they suffer from low efficiency due to tremendous computational cost and bulky memory usage. Recent efforts on compression GANs show noticeable progress in obtaining smaller generators by sacrificing image quality or involving a time-consuming searching process. In this work, we aim to address these issues by introducing a teacher network that provides a search space in which efficient network architectures can be found, in addition to performing knowledge distillation. First, we revisit the search space of generative models, introducing an inception-based residual block into generators. Second, to achieve target computation cost, we propose a one-step pruning algorithm that searches a student architecture from the teacher model and substantially reduces searching cost. It requires no 1 sparsity regularization and its associated hyper-parameters, simplifying the training procedure. Finally, we propose to distill knowledge through maximizing feature similarity between teacher and student via an index named Global Kernel Alignment (GKA). Our compressed networks achieve similar or even better image fidelity (FID, mIoU) than the original models with much-reduced computational cost, e.g., MACs. Code will be released at https://github.com/snap-research/CAT.

show abstract