Continuous Conditional Generative Adversarial Networks: Novel Empirical Losses and Label Input Mechanisms

Abstract

This work proposes the concept of continuous conditional generative adversarial network (CcGAN), the first generative model for image generation conditional on continuous, scalar conditions (termed regression labels). Existing conditional GANs (cGANs) are mainly designed for categorical conditions (e.g., class labels); conditioning on regression labels is mathematically distinct and raises two fundamental problems: (P1) Since there may be very few (even zero) real images for some regression labels, minimizing existing empirical versions of cGAN losses (a.k.a. empirical cGAN losses) often fails in practice; (P2) Since regression labels are scalar and infinitely many, conventional label input mechanisms (e.g., combining a hidden map of the generator/discriminator with a one-hot encoded label) are not applicable. To solve the above problems, by (S1) reformulating existing empirical cGAN losses to be appropriate for the continuous scenario; and (S2) proposing a naive label input (NLI) mechanism and an improved label input (ILI) mechanism to incorporate regression labels into the generator and the discriminator, we propose 4 CcGANs with employing different proposed losses and label input mechanisms. The reformulation in (S1) leads to two novel empirical discriminator losses, termed the hard vicinal discriminator loss (HVDL) and the soft vicinal discriminator loss (SVDL) respectively, and a novel empirical generator loss. The error bounds of the discriminator trained with HVDL and SVDL respectively are derived under mild assumptions in this work. To evaluate the performance of CcGANs, two new benchmark datasets (RC-49 and Cell-200) are proposed. A novel evaluation metric (Sliding Fr échet Inception Distance) is also proposed to replace Intra-FID when Intra-FID is not applicable. Our extensive experiments on several benchmark datasets (i.e., the Circular 2-D Gaussians, RC-49, UTKFace, Cell-200, and Steering Angle datasets) support the following observations: The proposed CcGAN is able to generate diverse, high-quality samples from the image distribution conditional on a given regression label; and CcGAN substantially outperforms cGAN both visually and quantitatively.

Continuous conditional generative adversarial networks for data-driven solutions of poroelasticity with heterogeneous material properties

“…With three examples presented, we observe that the ILI model performs better than the NLI for all cases except one particular case (Table 1). This finding is in good agreement with classification problems in Ding et al (2020). Ding et al (2020) speculates that the ILI overcomes the label inconsistency of the classification problems while the NLI could not.…”

“…This finding is in good agreement with classification problems in Ding et al (2020). Ding et al (2020) speculates that the ILI overcomes the label inconsistency of the classification problems while the NLI could not. Our reasoning for the outperformance of the INI stems from continuous conditional batch normalization, which can carry out temporal information more consistently than the element-wise addition in the NLI model.…”

“…In this study, we build a separate model for each primary variable (u h and p h ), although both primary variables can be trained together with a single model. A key aspect of this work is to apply the CcGAN image-to-image translation framework for timedependent PDEs by adapting the concept of a naive label input (NLI) and an improved label input (ILI) proposed by Ding et al (2020) to the framework developed by Kadeethum et al (accepted, 2021). The proposed framework in this work consists of a generator and critic where two types of architecture for the generator with a similar critic architecture are presented (Figure 2).…”

“…The second one adopts the ILI concept (i.e., ILI model) by injecting the temporal term to all layers inside the generator through conditional batch normalization (De Vries et al, 2017). However, in contrast to Ding et al (2020); De Vries et al ( 2017), our t n is not categorical data (i.e., not a tag of number ranging from zero to nine), but continuous variable (i.e., t n ∈ T. Hence, we replace embedded layers with an artificial neural network (ANN). To elaborate, each conditional batch normalization composes of one ANN and one batch normalization layer.…”

“…Recently, Ding et al (2020) developed continuous conditional generative adversarial networks (CcGAN) to condition the GAN model with continuous variables such as the weight of each animal. The CcGAN concept can extend the conditioned information from categorical data (e.g., cat or dog) to more quantitative measures (e.g., the weight of each animal).…”