Nonparametric Variational Auto-encoders for Hierarchical Representation Learning

Goyal, Prasoon; Hu, Zhiting; Liang, Xiaodan; Wang, Chenyu; Xing, Eric P.

doi:10.48550/arxiv.1703.07027

Cited by 15 publications

(12 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While AAE allows the prior to be arbitrary, how to select a prior that can best characterize the data distribution remains an open issue. Goyal et al [8] make an attempt to learn a non-parametric prior based on the nested Chinese restaurant process for VAEs. Learning is achieved by fitting it to the aggregated posterior distribution, which amounts to maximization of ELBO.…”

Section: Related Workmentioning

confidence: 99%

“…Inspired by these learned priors [8,9] for VAE, we propose in this paper the notion of code generators to learn a proper prior from data for AAE. The relations of our work with these prior arts are illustrated in Fig.…”

Section: Related Workmentioning

confidence: 99%

“…Some recent works [7][8][9] suggest that the choice of the prior may have a profound impact on the expressiveness of the model. As an example, in learning the VAE with a simple encoder and decoder, Hoffman and Johnson [7] conjecture that multimodal priors can achieve a higher variational lower bound on the data log-likelihood than is possible with the standard normal prior.…”

Section: Introductionmentioning

confidence: 99%

“…Tomczak and Welling [9] confirm the truth of this conjecture by showing that their multimodal prior, a mixture of the variational posteriors, consistently outperforms simple priors on a number of datasets in terms of maximizing the data loglikelihood. Taking one step further, Goyal et al [8] learn a tree-structured nonparametric Bayesian prior for capturing 1 National Chiao Tung University, 1001 Ta-Hsueh Rd., Hsinchu 30010, Taiwan the hierarchy of semantics presented in the data. All these priors are learned under the VAE framework following the principle of maximum likelihood.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Learning Priors for Adversarial Autoencoders

Wang¹,

Peng²,

Ko³

2019

Preprint

View full text Add to dashboard Cite

Most deep latent factor models choose simple priors for simplicity, tractability or not knowing what prior to use. Recent studies show that the choice of the prior may have a profound effect on the expressiveness of the model, especially when its generative network has limited capacity. In this paper, we propose to learn a proper prior from data for adversarial autoencoders (AAEs). We introduce the notion of code generators to transform manually selected simple priors into ones that can better characterize the data distribution. Experimental results show that the proposed model can generate better image quality and learn better disentangled representations than AAEs in both supervised and unsupervised settings. Lastly, we present its ability to do cross-domain translation in a text-to-image synthesis task.Keywords: Authors should not add keywords, as these will be chosen during the submission process (please refer to Section II (F) below for further details).

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Learning Priors for Adversarial Autoencoders

Wang¹,

Peng²,

Ko³

2019

Preprint

View full text Add to dashboard Cite

show abstract

“…Several works have studied more complex prior distributions, including learnable ones[83][84][85][86][87] 2. The objective eq.…”

mentioning

confidence: 99%

Differentiable Strong Lensing: Uniting Gravity and Neural Nets through Differentiable Probabilistic Programming

Chianese,

Coogan,

Hofma

et al. 2019

Preprint

View full text Add to dashboard Cite

The careful analysis of strongly gravitationally lensed radio and optical images of distant galaxies can in principle reveal DM (sub-)structures with masses several orders of magnitude below the mass of dwarf spheroidal galaxies. However, analyzing these images is a complex task, given the large uncertainties in the source and the lens. Here, we leverage and combine three important computer science developments to approach this challenge from a new perspective. (a) Convolutional deep neural networks, which show extraordinary performance in recognizing and predicting complex, abstract correlation structures in images. (b) Automatic differentiation, which forms the technological backbone for training deep neural networks and increasingly permeates 'traditional' physics simulations, thus enabling the application of powerful gradient-based parameter inference techniques. (c) Deep probabilistic programming languages, which not only allow the specification of probabilistic programs and automatize the parameter inference step, but also the direct integration of deep neural networks as model components. In the current work, we demonstrate that it is possible to combine a deconvolutional deep neural network trained on galaxy images as source model with a fully-differentiable and exact implementation of the gravitational lensing physics in a single probabilistic model. This does away with hyperparameter tuning for the source model, enables the simultaneous optimization of nearly one hundred source and lens parameters with gradient-based methods, and allows the use of efficient gradient-based Hamiltonian Monte Carlo posterior sampling techniques. We consider this work as one of the first steps in establishing differentiable probabilistic programming techniques in the particle astrophysics community, which have the potential to significantly accelerate and improve many complex data analysis tasks.

show abstract

Deep Learning‐Based Multiomics Data Integration Methods for Biomedical Application

Wen¹,

Zheng

Leng³

et al. 2023

Advanced Intelligent Systems

View full text Add to dashboard Cite

Driven by medical radiomics and high-throughput techniques, there has been explosive growth in the amount of advanced multiomics data in modern biomedical research. These data contain abundant information reflecting cellular context, allowing researchers to disentangle biomolecular mechanisms, and gain a more comprehensive understanding of biological processes related to complex diseases. [1,2] Single-omics data provide limited information on biological systems. The systematic study of biomedical objects by integrating and analyzing data at diverse scales is an emerging field. [3][4][5] The integration of multiomics data makes it possible to understand complex biological systems from different perspectives. [5,6] Using multiomics data can identify the best therapeutic targets and address key biomedical objects, including personalized complex disease therapy, [7][8][9][10][11] drug discovery, [12][13][14][15] and drug target discovery. [16][17][18][19][20] However, multiomics data are complex, high dimensional, and heterogeneous. [21,22] By far the most important challenge is how to extract valuable knowledge from these data. Since various omics data are obtained by different measurement techniques, the data distribution for different omics is different. The inconsistency of data distribution is one of the difficulties to be overcome. At the same time, each sample contains data from multiple omics, and exploring potential correlations between these data is also a problem. Biomedical data is also characterized by high dimensionality and few samples, which can generate the curse of dimensionality during data mining and reduce the generalization ability of the model. To address this challenge, researchers have developed methods such as multiple kernel learning, Bayesian approaches, dimensionality reduction approaches, network-based methods, and deep learning (DL) methods. [23][24][25][26][27][28][29][30] With the continuous development of DL, up to now, many DL methods have emerged to integrate multiomics data. The DL methods can be utilized as an efficient framework to process a large amount of multiomics, high-dimensional, and complex data. The DL-based methods can capture the typical nonlinearities and complex relationships in biological data to achieve more accurate predictions. DL, represented by multiomics data modeling, has achieved substantial success in biomedical fields. The combination of DL and multiomics data is significantly beneficial for the development of modern biomedical studies.In this article, we mainly focused on how to effectively extract omics data representations and how to integrate these representations. We classified multiomics integration models into six categories according to the framework of DL-based multiomics data integration methods. In addition, we reviewed and

show abstract

Nonparametric Variational Auto-encoders for Hierarchical Representation Learning

Cited by 15 publications

References 3 publications

Learning Priors for Adversarial Autoencoders

Learning Priors for Adversarial Autoencoders

Differentiable Strong Lensing: Uniting Gravity and Neural Nets through Differentiable Probabilistic Programming

Deep Learning‐Based Multiomics Data Integration Methods for Biomedical Application

Contact Info

Product

Resources

About