Generative Adversarial Networks (GANs) can be used in a wide range of applications where drawing samples from a data probability distribution without explicitly representing it is essential. Unlike the deep Convolutional Neural Networks (CNNs) trained for mapping an input to one of the multiple outputs, monitoring the overfitting and underfitting in GANs is not trivial since they are not classifying but generating a data. While training set and validation set accuracy give a direct sense of success in terms of overfitting and underfitting for CNNs during the training process, evaluating the GANs mainly depends on the visual inspection of the generated samples and generator/discriminator costs of the GANs. Unfortunately, visual inspection is far away of being objective and generator/discriminator costs are very nonintuitive. In this paper, a method was proposed for quantitatively determining the overfitting and underfitting in the GANs during the training process by calculating the approximate derivative of the Fréchet Distance between generated data distribution and real data distribution unconditionally or conditioned on a specific class. Both of the distributions can be obtained from the distribution of the embedding in the Discriminator Network of the GAN. The method is independent of the design architecture and the cost function of the GAN and empirical results on MNIST and CIFAR-10 support the effectiveness of the proposed method.
Summary
Fréchet inception distance (FID) has gained a better reputation as an evaluation metric for generative adversarial networks (GANs). However, it is subjected to fluctuation, namely, the same GAN model, when trained at different times can have different FID scores, due to the randomness of the weight matrices in the networks, stochastic gradient descent, and the embedded distribution (activation outputs at a hidden layer). In calculating the FIDs, embedded distribution plays the key role and it is not a trivial question from where obtaining it since it contributes to the fluctuation also. In this article, I showed that embedded distribution can be obtained from three different subspaces of the weight matrix, namely, from the row space, the null space, and the column space, and I analyzed the effect of the each space to Fréchet distances (FDs). Since the different spaces show different behaviors, choosing a subspace is not an insignificant decision. Instead of directly using the embedded distribution obtained from hidden layer's activations to calculate the FD, I proposed to use projection of embedded distribution onto the null space of the weight matrix among the three subspaces to avoid the fluctuations. My simulation results conducted at MNIST, CIFAR10, and CelebA datasets, show that, by projecting the embedded distributions onto the null spaces, possible parasitic effects coming from the randomness are being eliminated and reduces the number of needed simulations ≈25× in MNIST dataset, ≈21× in CIFAR10, and ≈12× in CelebA dataset.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.