Conditional density estimation and simulation through optimal transport

Tabak, Esteban G.; Trigila, Giulio; Zhao, Wenjun

doi:10.1007/s10994-019-05866-3

Cited by 8 publications

(5 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Additionally, most of these works are restricted to simple bivariate settings and/or single covariates. An optimal transport based approach was recently considered in Tabak et al (2020). Variational Bayes inference for a mixture model with multivariate normal component kernels with their means and the mixture probabilities varying with associated covariates has been considered in Dao and Tran (2021).…”

Section: Supplementary Materials Formentioning

confidence: 99%

Bayesian Semiparametric Covariate Informed Multivariate Density Deconvolution

Sarkar¹

2022

Preprint

View full text Add to dashboard Cite

Estimating the marginal and joint densities of the long-term average intakes of different dietary components is an important problem in nutritional epidemiology. Since these variables cannot be directly measured, data are usually collected in the form of 24-hour recalls of the intakes. The problem of estimating the density of the latent long-term average intakes from their observed but error contaminated recalls then becomes a problem of multivariate deconvolution of densities. The underlying densities could potentially vary with the subjects' demographic characteristics such as sex, ethnicity, age, etc. The problem of density deconvolution in the presence of associated precisely measured covariates has, however, never been considered before, not even in the univariate setting. We present a flexible Bayesian semiparametric approach to covariate informed multivariate deconvolution. Building on recent advances on copula deconvolution and conditional tensor factorization techniques, our proposed method not only allows the joint and the marginal densities to vary flexibly with the associated predictors but also allows automatic selection of the most influential predictors. Importantly, the method also allows the density of interest and the density of the measurement errors to vary with potentially different sets of predictors. We design Markov chain Monte Carlo algorithms that enable efficient posterior inference, appropriately accommodating uncertainty in all aspects of our analysis. The empirical efficacy of the proposed method is illustrated through simulation experiments. Its practical utility is demonstrated in the afore-described nutritional epidemiology applications in estimating covariate adjusted long term intakes of different dietary components. An important by-product of the approach is a solution to covariate informed ordinary multivariate density estimation. Supplementary materials include substantive additional details and R codes are also available online.

show abstract

Section: Supplementary Materials Formentioning

confidence: 99%

Bayesian Semiparametric Covariate Informed Multivariate Density Deconvolution

Sarkar¹

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Various error forms can be useful for applying dimension reduction in different inference tasks, as each inference task often has its "preferred" way to quantify the error. For example, the optimization problems in transport maps [9,40,49] and Stein variational methods [25,37] are formulated by KL divergence, tensor train [19] and other approximation methods, e.g., [36], gives bounds in Hellinger distance, and the min-max formulation in density estimation methods such as [52,53,55] rely on the estimation error in (10). Unless otherwise specified, we only consider the estimations error and statistical divergences of the full-dimensional approximate target densities π r (x), r = {f, g, l} rather than their reduced dimensional counterparts π r (x r ) in the rest of this paper.…”

Section: Accuracy Of Approximate Target Densitiesmentioning

confidence: 99%

A unified performance analysis of likelihood-informed subspace methods

Cui¹,

Tong²

2021

Preprint

View full text Add to dashboard Cite

The likelihood-informed subspace (LIS) method [e.g., Cui et al., Inverse Probl., 30 (2014), 114015] offers a viable route to reducing the dimensionality of high-dimensional probability distributions arisen in Bayesian inference. LIS identifies an intrinsic low-dimensional linear subspace where the target distribution differs the most from some tractable reference distribution. Such a subspace can be identified using the leading eigenvectors of a Gram matrix of the gradient of the log-likelihood function. Then, the original high-dimensional target distribution is approximated through various forms of ridge approximations of the likelihood function, in which the approximated likelihood only has support on the intrinsic low-dimensional subspace. This approximation enables the design of inference algorithms that can scale sub-linearly with the apparent dimensionality of the problem. Intuitively, the accuracy of the approximation, and hence the performance of the inference algorithms, are influenced by three factors-the dimension truncation error in identifying the subspace, Monte Carlo error in estimating the Gram matrices, and Monte Carlo error in constructing ridge approximations. This work establishes a unified framework to analysis each of these three factors and their interplay. Under mild technical assumptions, we establish error bounds for a range of existing dimension reduction techniques based on the principle of LIS. Our error bounds also provide useful insights into the accuracy comparison of these methods. In addition, we analyze the integration of LIS with sampling methods such as Markov Chain Monte Carlo (MCMC) and sequential Monte Carlo (SMC). We also demonstrate our analyses on a linear inverse problem with Gaussian prior, which shows that all the estimates can be dimension-independent if the prior covariance is a trace-class operator. Finally, we demonstrate various aspects of our theoretical claims on several nonlinear inverse problems.

show abstract

“…Our theoretical model is based on optimal transport, in particular on the barycenter of probability measures. The idea of applying barycenters to conditional density estimation originates from [37], while the application to latent variable discovery is based on the previous work in [36,43]. This paper lays the theoretical foundation for the technique of barycenters, and introduces several neural network-based algorithms.…”

Section: Related Workmentioning

confidence: 99%

“…The test function ψ Y (y)ψ Z (z) in (7) serves as the "discriminator" that checks that all the conditional distributions ρ(x|z) have been pushed-forward to the same barycenter µ. The technique of discriminator has appeared in [4] to train the generative adversarial networks, and it has been applied to the barycenter problem by [37], which derived test functions of the form ψ(y, z) such that ψ(y, z)dν(z) ≡ 0 Theorem 2 improves this technique, because ψ Y (y)ψ Z (z) has much less complexity than ψ(y, z). From a data-based perspective, with the distributions given through sample points {x i , y i , z i } N i=1 , the test function ψ(y, z) becomes a full N × N matrix, whereas ψ Y (y), ψ Z (z) are two 1 × N vectors, which can be seen as providing a rank-one factorization of ψ(y, z).…”

Section: Conditional Transport Mapsmentioning

confidence: 99%

See 1 more Smart Citation

Conditional Density Estimation, Latent Variable Discovery and Optimal Transport

Yang

Tabak

2019

Preprint

Self Cite

View full text Add to dashboard Cite

A framework is proposed that addresses both conditional density estimation and latent variable discovery. The objective function maximizes explanation of variability in the data, achieved through the optimal transport barycenter generalized to a collection of conditional distributions indexed by a covariate -either given or latent-in any suitable space. Theoretical results establish the existence of barycenters, a minimax formulation of optimal transport maps, and a general characterization of variability via the optimal transport cost. This framework leads to a family of non-parametric neural network-based algorithms, the BaryNet, with a supervised version that estimates conditional distributions and an unsupervised version that assigns latent variables. The efficacy of BaryNets is demonstrated by tests on both artificial and real-world data sets. A parallel drawn between autoencoders and the barycenter framework leads to the Barycentric autoencoder algorithm (BAE).

show abstract

Conditional density estimation and simulation through optimal transport

Cited by 8 publications

References 22 publications

Bayesian Semiparametric Covariate Informed Multivariate Density Deconvolution

Bayesian Semiparametric Covariate Informed Multivariate Density Deconvolution

A unified performance analysis of likelihood-informed subspace methods

Conditional Density Estimation, Latent Variable Discovery and Optimal Transport

Contact Info

Product

Resources

About