HGAN: Hyperbolic Generative Adversarial Network

Abstract

Recently, Hyperbolic Spaces in the context of Non-Euclidean Deep Learning have gained popularity because of their ability to represent hierarchical data. We propose that it is possible to take advantage of the hierarchical characteristic present in the images by using hyperbolic neural networks in a GAN architecture. In this study, different configurations using fully connected hyperbolic layers in the GAN, WGAN, CGAN, and the mapping network of the StyleGAN2 are tested in what we call the HGAN, HWGAN, HCGAN, and HStyleGAN, respectively. Furthermore, we test multiple values of curvature and introduce an exponential way to train it. The results are measured using the Inception Score (IS) and the Fréchet Inception Distance (FID) over the MNIST dataset and with FID over CIFAR-10. Depending on the configuration and space curvature, better results are achieved for each proposed hyperbolic version than their euclidean counterpart.

Hyperbolic Deep Learning in Computer Vision: A Survey

“…Using the intuition that images are organized hierarchically, several works have proposed hyperbolic generative adversarial networks (GANs). Lazcano et al (2021) propose a hyperbolic GAN which replaces some of the Euclidean layers in both the generator and discriminator with hyperbolic layers (Ganea et al, 2018a) with learnable curvature. Lazcano et al (2021) propose hyperbolic variants of the original GAN (Goodfellow et al, 2020), the Wasserstein GAN WGAN-GP (Gulrajani et al, 2017) and conditional GAN CGAN (Mirza & Osindero, 2014).…”

“…Lazcano et al (2021) propose a hyperbolic GAN which replaces some of the Euclidean layers in both the generator and discriminator with hyperbolic layers (Ganea et al, 2018a) with learnable curvature. Lazcano et al (2021) propose hyperbolic variants of the original GAN (Goodfellow et al, 2020), the Wasserstein GAN WGAN-GP (Gulrajani et al, 2017) and conditional GAN CGAN (Mirza & Osindero, 2014). The paper finds that their best configurations of Euclidean and hyperbolic layers generally improved the Inception Score (Salimans et al, 2016) and Frechet Inception Distance (Heusel et al, 2017) on MNIST image generation, with the best improvements in the GAN architecture.…”

“…In the wake of other disciplines, computer vision has in recent years also benefited from research into deep learning in hyperbolic space. A quickly growing body of literature has shown that hyperbolic embeddings benefit few-shot learning (Fang et al, 2021;Khrulkov et al, 2020;Gao et al, 2021;Guo et al, 2022), zero-shot recognition (Long et al, 2020;Liu et al, 2020;Ghadimi Atigh et al, 2021;Hong et al, 2023b), out-of-distribution generalization (Khrulkov et al, 2020;Hong et al, 2023a;Guo et al, 2022), uncertainty quantification (Khrulkov et al, 2020;Ghadimi Atigh et al, 2022;Chen et al, 2022), generative learning (Kingma & Welling, 2013;Rezende et al, 2014;Lazcano et al, 2021;Heusel et al, 2017), and hierarchical representation learning (Dhall et al, 2020;Long et al, 2020;Gulshad et al, 2023;Liu et al, 2020;Ghadimi Atigh et al, 2022) amongst others. These works show evidence that hyperbolic geometry has a lot of potential for learning in computer vision.…”

Hyperbolic Geometry in Computer Vision: A Novel Framework for Convolutional Neural Networks

“…This approach involves projecting Euclidean features onto the hyperbolic space in the task heads and designing loss functions based on hyperbolic geometry. Such simple hybrid architectures have been proven effective in various computer vision tasks like recognition [15,20,28], segmentation [1,18], reconstruction/generation [25,31,34,36,40], and metric learning [10,46]. However, Guo et al [15] have shown that learning a mixture of Euclidean and hyperbolic features can exacerbate gradient vanishing, and it remains unclear whether these hybrid models can fully exploit the properties of hyperbolic geometry.…”

Hyperbolic disc embedding of functional human brain connectomes using resting-state fMRI

“…Space of hyperbolic geometry is in fact, just the place we live in, as indicated by the special relativity with multi-layered, hierarchical and multiplex-correlated way. The most characteristic feature about the hyperbolic space against other types of space is that it best represents the data with hierarchical structure (Lazcano et al, 2021), as in deep neural network implemented in deep learning (Peng et al, 2021). Prior research about embedding into non-Euclidean spaces also reports the hyperbolic space as the matching geometry to embed tree-like structures, employing the least distortion (Gu et al, 2018;Sala et al, 2018).…”