Generating Data using Monte Carlo Dropout

Abstract

For many analytical problems the challenge is to handle huge amounts of available data. However, there are data science application areas where collecting information is difficult and costly, e.g., in the study of geological phenomena, rare diseases, faults in complex systems, insurance frauds, etc. In many such cases, generators of synthetic data with the same statistical and predictive properties as the actual data allow efficient simulations and development of tools and applications. In this work, we propose the incorporation of Monte Carlo Dropout method within Autoencoder (MCD-AE) and Variational Autoencoder (MCD-VAE) as efficient generators of synthetic data sets. As the Variational Autoencoder (VAE) is one of the most popular generator techniques, we explore its similarities and differences to the proposed methods. We compare the generated data sets with the original data based on statistical properties, structural similarity, and predictive similarity. The results obtained show a strong similarity between the results of VAE, MCD-VAE and MCD-AE; however, the proposed methods are faster and can generate values similar to specific selected initial instances.

Multiple Imputation for Biomedical Data using Monte Carlo Dropout Autoencoders

“…Since its introduction by Gal and Ghahramani in 2016 [6], the Monte Carlo Dropout (MCD) method has been implemented within various neural networks architectures, like convolutional and recurrent networks [24,25]. The properties of dropout [26] allowed its implementations within different machine learning prediction tasks [27,28,18]. In this work, we apply the MCD within the AE and VAE decoder layers in order to obtain multiple generated inputs.…”

“…By averaging the output values for a specific missing value, we achieve better imputation accuracy than by using the classical approaches. The implementation of MCD within the AE and VAE (MCD-(V)AE) was first time proposed by [18] with the intention to improve subject specific generation from AE and VAE models. The architecture of this method is described in Fig1.…”

“…The idea is to construct a model that absorbs most information from the data and uses dimensionality reduction property of AE or VAE to reduce the noise of random missingness. As shown in [18] the advantage of MCD-(V)AE model over other autoencoders is that it captures the specific distribution of the provided observations from which the generation is done. Hence, not relying on all the data can improve imputation error and better preserve true relationship among the attributes.…”

“…For instance, medGAN has been proposed to generate synthetic health care patient records [14], the GAN has been used to generate synthetic passenger name records [15], and Generative Adversarial Imputation Nets (GAIN) impute missing components conditioned on what is actually observed [16]. Different types of variational autoencoders have been proposed as data generators: from the text generating tasks [17] to the tasks where generated data shall preserve the distributional properties of the original data [18]. A method that enables a variational autoencoder to handle incomplete heterogenous data was proposed in [19] while the model introduced by [20] achieves multiple imputation based on overcomplete deep denoising autoencoders.…”

To BAN or Not to BAN: Bayesian Attention Networks for Reliable Hate Speech Detection

“…MCD was recently used within several models and different architectures to obtain the prediction uncertainty and improve the classification results [49][50][51]. Transformer networks were not yet analyzed.…”