Determining overfitting and underfitting in generative adversarial networks using Fréchet distance

Abstract

Generative Adversarial Networks (GANs) can be used in a wide range of applications where drawing samples from a data probability distribution without explicitly representing it is essential. Unlike the deep Convolutional Neural Networks (CNNs) trained for mapping an input to one of the multiple outputs, monitoring the overfitting and underfitting in GANs is not trivial since they are not classifying but generating a data. While training set and validation set accuracy give a direct sense of success in terms of overfitting and underfitting for CNNs during the training process, evaluating the GANs mainly depends on the visual inspection of the generated samples and generator/discriminator costs of the GANs. Unfortunately, visual inspection is far away of being objective and generator/discriminator costs are very nonintuitive. In this paper, a method was proposed for quantitatively determining the overfitting and underfitting in the GANs during the training process by calculating the approximate derivative of the Fréchet Distance between generated data distribution and real data distribution unconditionally or conditioned on a specific class. Both of the distributions can be obtained from the distribution of the embedding in the Discriminator Network of the GAN. The method is independent of the design architecture and the cost function of the GAN and empirical results on MNIST and CIFAR-10 support the effectiveness of the proposed method.

An Efficient High-Throughput Screening of High Gentamicin-Producing Mutants Based on Titer Determination Using an Integrated Computer-Aided Vision Technology and Machine Learning

“…These results suggest that an ensemble algorithm could provide a higher prediction performance than a single approach . In addition, the overfitting and underfitting models will lead to an increased generalization error in machine learning. , Then, the learning curves were drawn to evaluate the ability in different amounts of data. The error decreases as the number of samples increases by contrasting the learning curve scores (10-fold cross-validation), where PLS, RF, and GBR could converge on a higher score level compared to SVM (underfit), especially RF, which has the highest R 2 (the number of training samples = 712, training R 2 = 0.998, testing R 2 = 0.982), as shown in Figure c–h.…”

An efficient high-throughput screening of high gentamicin-producing mutants based on titer determination using an integrated computer-aided vision technology and machine learning

“…In machine learning, overfitting and underfitting models will lead to the increase in generalization error (Eken, 2021;Salam, Azar, Elgendy, & Fouad, 2021). RF could be balanced by adjusting parameters and the prediction power for unknown samples would be fully realized (Hou et al, 2021;Torre-Tojal, Bastarrika, Boyano, Lopez-Guede, & Graña, 2022).…”