Variations in Variational Autoencoders - A Comparative Evaluation

Abstract

Variational Auto-Encoders (VAEs) are deep latent space generative models which have been immensely successful in many applications such as image generation, image captioning, protein design, mutation prediction, and language models among others. The fundamental idea in VAEs is to learn the distribution of data in such a way that new meaningful data can be generated from the encoded distribution. This concept has led to tremendous research and variations in the design of VAEs in the last few years creating a field of its own, referred to as unsupervised representation learning. This paper provides a muchneeded comprehensive evaluation of the variations of the VAEs based on their end goals and resulting architectures. It further provides intuition as well as mathematical formulation and quantitative results of each popular variation, presents a concise comparison of these variations, and concludes with challenges and future opportunities for research in VAEs.

Recent Advances in Variational Autoencoders With Representation Learning for Biomedical Informatics: A Survey

“…Despite the above-mentioned advantages of VAEs, they do have some premise constraints such as compared to GANs, the samples it generates tend to be blurry and of lower quality [109]. In order to solve the problems, researchers have proposed many variations of the VAEs based on different task requirements such as representation learning, disentanglement and deep clustering with the goal of greatly improving the intra-class variations and quality of the generated data [110]. The ability of VAEs to synthesize images at stateof-art levels gives hope that the chronic scarcity of labeled data in the biomedical field can be resolved.…”

Stacking and Chaining of Normalization Methods in Deep Learning-Based Classification of Colorectal Cancer Using Gut Microbiome Data

“…Therefore, the proposed stacking technique is a form of data augmentation. While conventional data augmentation methods first identify a distribution and then randomly generate data samples within that distribution [53], the proposed method applies several normalisation methods on existing data and then combines results that have matching distributions.…”

Integrated Classification and Localization of Targets Using Bayesian Framework In Automotive Radars

“…Both, the triplet loss and triplet mining are computed online based on semi-hard examples [12]. As VAE is trained to maximize the likelihood of a data-point for a given estimate of latent code using a KL-divergence loss L KL [13]. As a result, the final loss for a given triplet-input is summarized as a weighted sum of the reconstruction, KLdivergence and triplet loss (α rec L rec +α trip L triplet +L KL ) [11], where the choices of weighting hyperparameters are computed during cross-validation.…”