Deep Feature Consistent Variational Autoencoder

Hou, Xianxu; Shen, Linlin; Sun, Ke; Qiu, Guoping

doi:10.1109/wacv.2017.131

Cited by 271 publications

(221 citation statements)

References 30 publications

Supporting

Mentioning

214

Contrasting

Unclassified

Order By: Relevance

“…This is known as the reparameterization trick and allows the calculation of the gradient of the loss function with respect to the parameters of this architecture [56]. VAEs are known to produce blurry reconstructions [57,58] of the inputs, but more meaningful data representations, unlike vanilla AEs. That is because the VAEs are forced to minimize reconstruction errors from feature vectors sampled from the latent distribution.…”

Section: Unsupervised Learningmentioning

confidence: 99%

ML4Chem: A Machine Learning Package for Chemistry and Materials Science

Khatib

Jong²

2020

Preprint

View full text Add to dashboard Cite

ML4Chem is an open-source machine learning library for chemistry and materials science. It provides an extendable platform to develop and deploy machine learning models and pipelines and is targeted to the non-expert and expert users. ML4Chem follows user-experience design and offers the needed tools to go from data preparation to inference. Here we introduce its atomistic module for the implementation, deployment, and reproducibility of atom-centered models. This module is composed of six core building blocks: data, featurization, models, model optimization, inference, and visualization. We present their functionality and easiness of use with demonstrations utilizing neural networks and kernel ridge regression algorithms. * melkhatibr@lbl.gov arXiv:2003.13388v1 [physics.chem-ph] 2 Mar 2020 2 external programs, and exportation of any of its modules' outputs. Also, ML4Chem is in its infancy bringing up the possibility to shape its future directions based on current users' needs and ML paradigms.Here we introduce the atomistic module where ML algorithms learn underlying relationships between molecules and properties treating atoms as central objects. They exploit the principle of locality in Physics: a global quantity is defined as a sum that runs over many localized contributions. These localized contributions usually account for interactions of an atom and its nearest-neighbor atoms (many-body interactions). Atomistic models are very useful and have been successfully applied for the acceleration of molecular dynamics simulations [17][18][19], identification of phase transitions in materials [20], determination of energy and atomic forces with high accuracy [21,22], the search of saddle-points[23] and the prediction of atomic charges [24,25].This publication is organized as follows: in section II, we will discuss the design and architecture of ML4Chem's atomistic module. Each of its core blocks is introduced in Section III and we will demonstrate the code's capabilities through a series of demonstration examples in Section IV. Finally, conclusions and perspectives are drawn. II. ATOMISTIC MODULE: DESIGN AND ARCHITECTURE ML4Chem and its modules are written in Python in an object-oriented programming paradigm and are built on top of popular open-source projects to avoid duplication of efforts. In this regard, all deep learning computations are implemented with Pytorch[3]. Mathematical and linear algebra operations are executed by Numpy [26] or Scipy [27,28] that are widely used and recognized for this purpose. Parallelism is achieved with a flexible library for parallel computing called Dask [29]. Dask enables computational scaling-up from a laptop to High-Performance Computing (HPC) clusters effortlessly and offers a web dashboard to real-time monitoring. This is particularly valuable because it provides a good estimation to users about the status of calculations, and helps at profiling computations. Good documentation is another important aspect, as the lack of it can harm usability. ML4Chem's source code is docum...

show abstract

Section: Unsupervised Learningmentioning

confidence: 99%

ML4Chem: A Machine Learning Package for Chemistry and Materials Science

Khatib

Jong²

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Deep Recurrent Attentive Writer (DRAW) [29] combines spatial attention mechanism with a sequential variational autoencoding framework that allows iterative generation of images. [30] and [17] consider replacing per-pixel loss with perceptual similarities using either multi-scale structural similarity score or a perceptual loss based on deep features extracted from pretrained deep networks.…”

Section: Variational Autoencodermentioning

confidence: 99%

“…Following works [19,31,32,33,34,35,36,37,38,39,40,41] [17] and Wasserstein GAN (WGAN) [19] to improve the perceptual quality of the output images generated by VAE and enhance the effectiveness of VAE representations for semi-supervised learning. In addition, a combination of VAE and GAN was also proposed by [42].…”

Section: Generative Adversarial Networkmentioning

confidence: 99%

“…The 19-layer VGGNet [3] is chosen as loss network Φ Figure 3: Face images generated from 100-dimension latent vector z ∼ N (0, 1) by different models. We compare our VAE-WGAN with DCGAN [32], WGAN [19], VAE/GAN [42], LSGAN [41], Plain-VAE [16] and DFC-VAE [17].…”

Section: Training Detailsmentioning

confidence: 99%

“…As shown in Figure 3 and Figure 5: Face images reconstructed by different models. We compare our VAE-WGAN with Plain-VAE [16], VAE/GAN [42] and DFC-VAE [17]. Figure 6: CIFAR images reconstructed by different models.…”

Section: Arbitrary Image Generationmentioning

confidence: 99%

See 2 more Smart Citations

Improving variational autoencoder with deep feature consistent and generative adversarial training

et al. 2019

Self Cite

View full text Add to dashboard Cite

We present a new method for improving the performances of variational autoencoder (VAE). In addition to enforcing the deep feature consistent principle thus ensuring the VAE output and its corresponding input images to have similar deep features, we also implement a generative adversarial training mechanism to force the VAE to output realistic and natural images. We present experimental results to show that the VAE trained with our new method outperforms state of the art in generating face images with much clearer and more natural noses, eyes, teeth, hair textures as well as reasonable backgrounds. We also show that our method can learn powerful embeddings of input face images, which can be used to achieve facial attribute manipulation. Moreover we propose a multi-view feature extraction strategy to extract effective image representations, which can be used to achieve state of the art performance in facial attribute prediction.recovery [8,9,10], image privacy protection [11], unsupervised dimension reduction [12] and many other applications [13,14,15]. Deep convolutional generative models, as a branch of unsupervised learning technique in machine learning, have become an area of active research in recent years. A generative model trained with a given image database can be useful in several ways. One is to learn the essence of a dataset and generate realistic images similar to those in the dataset from random inputs. The whole dataset is "compressed" into the learned parameters of the model, which are significantly smaller than the size of the training dataset. The other is to learn reusable feature representations from unlabeled image datasets for a variety of supervised learning tasks such as image classification.In this paper, we propose a new method to train the variational autoencoder (VAE)[16] to improve its performance. In particular, we seek to improve the quality of the generated images to make them more realistic and less blurry. To achieve this, we employ objective functions based on deep feature consistent principle [17] and generative adversarial network [18,19] instead of the problematic per-pixel loss functions. The deep feature consistent can help capture important perceptual features such as spatial correlation through the learned convolutional operations, while the adversarial training helps to produce images that reside on the manifold of natural images. We also introduce several techniques to improve the convergence of GAN training in this context. In particular, instead of directly using the generated images and the real images in pixel space, the corresponding deep features extracted from pretrained networks are used to train the generator and the discriminator network. We also propose to further relax the constraint on the output of the discriminator network to balance the image reconstruction loss and the adversarial loss. We present experimental results to show that our new method can generate face images with much clearer facial parts such as eyes, nose, mouth, teeth, ears and hair textur...

show abstract

Good practices for Bayesian optimization of high dimensional structured spaces

et al. 2021

View full text Add to dashboard Cite

The increasing availability of structured but high dimensional data has opened new opportunities for optimization. One emerging and promising avenue is the exploration of unsupervised methods for projecting structured high dimensional data into low dimensional continuous representations, simplifying the optimization problem and enabling the application of traditional optimization methods. However, this line of research has been purely methodological with little connection to the needs of practitioners so far. In this article, we study the effect of different search space design choices for performing Bayesian optimization in high dimensional structured datasets. In particular, we analyses the influence of the dimensionality of the latent space, the role of the acquisition function and evaluate new methods to automatically define the optimization bounds in the latent space. Finally, based on experimental results using synthetic and real datasets, we provide recommendations for the practitioners.

show abstract

Deep Feature Consistent Variational Autoencoder

Cited by 271 publications

References 30 publications

ML4Chem: A Machine Learning Package for Chemistry and Materials Science

ML4Chem: A Machine Learning Package for Chemistry and Materials Science

Improving variational autoencoder with deep feature consistent and generative adversarial training

Good practices for Bayesian optimization of high dimensional structured spaces

Contact Info

Product

Resources

About