Conditional Density Estimation, Latent Variable Discovery and Optimal Transport

Abstract

A framework is proposed that addresses both conditional density estimation and latent variable discovery. The objective function maximizes explanation of variability in the data, achieved through the optimal transport barycenter generalized to a collection of conditional distributions indexed by a covariate -either given or latent-in any suitable space. Theoretical results establish the existence of barycenters, a minimax formulation of optimal transport maps, and a general characterization of variability via the optimal transport cost. This framework leads to a family of non-parametric neural network-based algorithms, the BaryNet, with a supervised version that estimates conditional distributions and an unsupervised version that assigns latent variables. The efficacy of BaryNets is demonstrated by tests on both artificial and real-world data sets. A parallel drawn between autoencoders and the barycenter framework leads to the Barycentric autoencoder algorithm (BAE).

Distributional barycenter problem through data-driven flows

“…The methodology for the solution of the data-driven distributional barycenter problem proposed in this article can be used with general cost functions. It improves significantly over previous approaches to the barycenter problem based on adversarial games ( [7,28,34]). The latter have two players: one that proposes cost-minimizing maps through time-evolving flows, and another that builds test functions to enforce the push-forward condition.…”

“…Optimal transport and the related Wasserstein barycenter problem have undergone rapid development during the last ten years, with a particular focus on applications to the analysis of data and machine learning [11], ranging from gene expression [24] to economics [9]. Procedures based on optimal transport have been used for density and conditional density estimation [30,28], data augmentation [19], image classification [12,31,35], computer vision [25,32,2,20], factor discovery [34] and data imputation [26].…”

Conditional Sampling With Monotone GANs

“…Since the optimal discriminator gives the Jensen-Shannon divergence up to a constant, this choice is valid in (2). Indeed, as shown in [32], a variety of divergences may be tightly lower-bounded by similar min-max functionals-a more general feature of dual formulations of measure transport problems [47]. We approximate the components K and F of T with neural networks and replace the strict monotonicity constraint on T with an average monotonicity constraint.…”