Clustering, factor discovery and optimal transport

Abstract

The clustering problem, and more generally, latent factor discovery -or latent space inference-is formulated in terms of the Wasserstein barycenter problem from optimal transport. The objective proposed is the maximization of the variability attributable to class, further characterized as the minimization of the variance of the Wasserstein barycenter. Existing theory, which constrains the transport maps to rigid translations, is extended to affine transformations. The resulting non-parametric clustering algorithms include k-means as a special case and exhibit more robust performance. A continuous version of these algorithms discovers continuous latent variables and generalizes principal curves. The strength of these algorithms is demonstrated by tests on both artificial and real-world data sets.

Conditional Density Estimation, Latent Variable Discovery and Optimal Transport

“…A simple approach is to take the maen variability of the conditional distributions ρ(x|z). Yet this approach does not always lead to sensible results: the k-means algorithm, for example, minimizes the sum of squared errors, or equivalently the weighted average of each cluster's variance, and it is known that it often fails to recognize clusters with different sizes and shapes [43].…”

“…Our theoretical model is based on optimal transport, in particular on the barycenter of probability measures. The idea of applying barycenters to conditional density estimation originates from [37], while the application to latent variable discovery is based on the previous work in [36,43]. This paper lays the theoretical foundation for the technique of barycenters, and introduces several neural network-based algorithms.…”

“…Remark 5. In prior work, optimal transport barycenter has been applied to the clustering problem in [36,43], which study the case with squared Euclidean distance cost and solve directly the primal problem min…”

“…If we simplify the transport maps T k by their first-moment approximations (15), then [36] shows that (19) produces the k-means algorithm. If we approximate T k so that it aligns the second moments of ρ(x|z) and µ, then [43] shows that (19) leads to more robust algorithms that recognize non-isotropic clusters. While [36,43] only compute the membership probabilities p(k|x i ), the dual problem (18) solves for both p(k|x i ) and T k , without any simplifying assumption on T k .…”

“…K}, the classical autoencoder's reconstruction loss reduces to the sum of within-cluster variances and the algorithm becomes k-means. Yet, [43] shows empirically that minimizing the barycenter's variance leads to more robust algorithms than k-means.…”