An Overview of Variational Autoencoders for Source Separation, Finance, and Bio-Signal Applications

Abstract

Autoencoders are a self-supervised learning system where, during training, the output is an approximation of the input. Typically, autoencoders have three parts: Encoder (which produces a compressed latent space representation of the input data), the Latent Space (which retains the knowledge in the input data with reduced dimensionality but preserves maximum information) and the Decoder (which reconstructs the input data from the compressed latent space). Autoencoders have found wide applications in dimensionality reduction, object detection, image classification, and image denoising applications. Variational Autoencoders (VAEs) can be regarded as enhanced Autoencoders where a Bayesian approach is used to learn the probability distribution of the input data. VAEs have found wide applications in generating data for speech, images, and text. In this paper, we present a general comprehensive overview of variational autoencoders. We discuss problems with the VAEs and present several variants of the VAEs that attempt to provide solutions to the problems. We present applications of variational autoencoders for finance (a new and emerging field of application), speech/audio source separation, and biosignal applications. Experimental results are presented for an example of speech source separation to illustrate the powerful application of variants of VAE: VAE, β-VAE, and ITL-AE. We conclude the paper with a summary, and we identify possible areas of research in improving performance of VAEs in particular and deep generative models in general, of which VAEs and generative adversarial networks (GANs) are examples.

A Case for Synthetic Data in Regulatory Decision‐Making in Europe

“…Synthetic data are generated using several methods categorized as supervised or unsupervised algorithms. Deep learning (DL) techniques, such as GANs, or variational autoencoders (VAEs), as well as machine learning techniques, such as tree synthesizers or non‐machine learning techniques such as Gaussian copulas, all produce synthetic data modeled from a real dataset 4,16–18 . Machine learning methods, such as GAN models, create networks that are trained on the source “original” data to synthesize data by generating realistic data points similar to the original real data 19 .…”

Applications of Deep Learning for Drug Discovery Systems with BigData

“…For example, a principal components analysis (PCA) plot reduces the high-dimensional (D) data to 2D or 3D, and the data are visualized by plotting the newly obtained feature values on the x-axis and the y-axis, as well as the z-axis (dimensionality reduction) [92][93][94][95][96][97]. In addition, multilayered autoencoders are also used for dimensionality reduction and are a type of deep learning that compress data that are "recoverably" used for learning [98]. For example, when high-dimensional multi-omics data with tens of thousands of feature values are input, these feature values are added together after they are weighted differently, and then a new feature value is generated through nonlinear transformation using an activation function in a hidden layer (encoder).…”

Evaluation of Synthetic Data Generation Techniques in the Domain of Anonymous Traffic Classification

“…Since autoencoders attempt to train an encoder and decoder with as little loss as possible, they are susceptible to overfitting. Variational Autoencoders address this concern by encoding the input as a distribution over the latent space [36]. After a VAE is trained it can be used to generate new synthetic samples for a dataset.…”