RGAN: Rényi Generative Adversarial Network

Sarraf, Aydin; Nie, Yimin

doi:10.1007/s42979-020-00403-9

Cited by 9 publications

(12 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note that RényiGAN is based on a different Rényi cross-entropy definition 2 from the one used in RGAN (Sarraf & Nie, 2021). Hence, unlike what is claimed in Sarraf and Nie (2021) by referencing the preprint (Bhatia, Paul, Alajaji, Gharesifard, & Burlina, 2020) of this letter, the RGAN's loss function does not generalize the RényiGAN loss functions presented here. Moreover unlike RényiGAN, the RGAN loss function does not preserve the GANs theoretical result that the optimal generator for an optimal discriminator induces a probability distribution equal to the true Shannon divergence as α approaches 1, differs from an earlier namesake measure introduced in He et al (2003) and Hamza and Krim (2003) and defined using the Rényi entropy.…”

Section: Contributionsmentioning

confidence: 86%

“…U n c o r r e c t e d P r o o f data set distribution. Using a similar stability analysis to the one carried out for RGANs in Sarraf and Nie (2021), we derive the absolute condition number of our Rényi cross-entropy measure and show that the RényiGAN's generator loss function is stable for α ≥ 2. This result complements the one derived in Sarraf and Nie (2021), where stability of the RGAN loss function is shown for sufficiently small values of α.…”

Section: Contributionsmentioning

confidence: 99%

Section: Contributionsmentioning

confidence: 99%

“…In a concurrent work (Sarraf & Nie, 2021), a Rényi-based GAN is developed, called RGAN, by also using a Rényi cross-entropy to generalize the original GAN loss function, and it is shown that the resulting loss function is stable in terms of its absolute condition number (calculated using functional derivatives). Note that RényiGAN is based on a different Rényi cross-entropy definition 2 from the one used in RGAN (Sarraf & Nie, 2021). Hence, unlike what is claimed in Sarraf and Nie (2021) by referencing the preprint (Bhatia, Paul, Alajaji, Gharesifard, & Burlina, 2020) of this letter, the RGAN's loss function does not generalize the RényiGAN loss functions presented here.…”

Section: Contributionsmentioning

confidence: 99%

“…Loss function stability. We close this section by determining the absolute condition number (Sarraf & Nie, 2021) for our Rényi cross-entropy functional H α (•; •) used in the RényiGAN generator loss function in equation 4.2. Such a quantity may be a useful to measure loss function stability; in this sense, the loss function is called "stable" if small changes in its argument lead to small changes in its absolute condition number.…”

Section: Theorem 5 Consider Problems 43 and 44 For Training The Discriminator And Generator Neural Network Respectively Thenmentioning

confidence: 99%

See 4 more Smart Citations

Least kth-Order and Rényi Generative Adversarial Networks

et al. 2021

View full text Add to dashboard Cite

We investigate the use of parameterized families of information-theoretic measures to generalize the loss functions of generative adversarial networks (GANs) with the objective of improving performance. A new generator loss function, least kth-order GAN (LkGAN), is introduced, generalizing the least squares GANs (LSGANs) by using a kth-order absolute error distortion measure with k≥1 (which recovers the LSGAN loss function when k=2). It is shown that minimizing this generalized loss function under an (unconstrained) optimal discriminator is equivalent to minimizing the kth-order Pearson-Vajda divergence. Another novel GAN generator loss function is next proposed in terms of Rényi cross-entropy functionals with order α>0, α≠1. It is demonstrated that this Rényi-centric generalized loss function, which provably reduces to the original GAN loss function as α→1, preserves the equilibrium point satisfied by the original GAN based on the Jensen-Rényi divergence, a natural extension of the Jensen-Shannon divergence. Experimental results indicate that the proposed loss functions, applied to the MNIST and CelebA data sets, under both DCGAN and StyleGAN architectures, confer performance benefits by virtue of the extra degrees of freedom provided by the parameters k and α, respectively. More specifically, experiments show improvements with regard to the quality of the generated images as measured by the Fréchet inception distance score and training stability. While it was applied to GANs in this study, the proposed approach is generic and can be used in other applications of information theory to deep learning, for example, the issues of fairness or privacy in artificial intelligence.

show abstract

Section: Contributionsmentioning

confidence: 86%

Section: Contributionsmentioning

confidence: 99%

Section: Contributionsmentioning

confidence: 99%

Section: Contributionsmentioning

confidence: 99%

Section: Theorem 5 Consider Problems 43 and 44 For Training The Discriminator And Generator Neural Network Respectively Thenmentioning

confidence: 99%

See 3 more Smart Citations

Least kth-Order and Rényi Generative Adversarial Networks

et al. 2021

View full text Add to dashboard Cite

show abstract

Artificial Immune Cell, AI‐cell, a New Tool to Predict Interferon Production by Peripheral Blood Monocytes in Response to Nucleic Acid Nanoparticles

Chandler

Jain

Halman

et al. 2022

Small

View full text Add to dashboard Cite

Nucleic acid nanoparticles, or NANPs, rationally designed to communicate with the human immune system, can offer innovative therapeutic strategies to overcome the limitations of traditional nucleic acid therapies. Each set of NANPs is unique in their architectural parameters and physicochemical properties, which together with the type of delivery vehicles determine the kind and the magnitude of their immune response. Currently, there are no predictive tools that would reliably guide the design of NANPs to the desired immunological outcome, a step crucial for the success of personalized therapies. Through a systematic approach investigating physicochemical and immunological profiles of a comprehensive panel of various NANPs, the research team developes and experimentally validates a computational model based on the transformer architecture able to predict the immune activities of NANPs. It is anticipated that the freely accessible computational tool that is called an “artificial immune cell,” or AI‐cell, will aid in addressing the current critical public health challenges related to safety criteria of nucleic acid therapies in a timely manner and promote the development of novel biomedical tools.

show abstract