Training GAN with predictive centripetal acceleration

Abstract

To alleviate the issue of limit cycle behavior in training generative adversarial networks (GAN), in this paper, we draw inspiration from the centripetal acceleration algorithm and the modified predictive method (MPM) by Liang and Stokes (2019). Building upon geometric observation of uniform circular motion, we propose the predictive centripetal acceleration algorithm (PCA). First and foremost, we prove the last-iterate convergence of the PCA on the bilinear game, which is a special case of the GAN. Besides, by combining PCA with the stochastic gradient descent (SGD) algorithm and adaptive moment estimation (Adam) algorithm, we propose two variants, which are called stochastic PCA (SPCA) and PCA-Adam, for the practical training GAN. Last but not least, experiments conducted on the bilinear game, multivariate Gaussian distribution, and CIFAR10 and CelebA datasets validate the effectiveness of the proposed algorithm.