Parameter Estimation in Finite Mixture Models by Regularized Optimal Transport: A Unified Framework for Hard and Soft Clustering

Dessein, Arnaud; Papadakis, Nicolas; Deledalle, Charles-Alban

doi:10.48550/arxiv.1711.04366

Cited by 7 publications

(11 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many methods that combine OT and clustering [34,35,36,37] focus on using the Wasserstein distance to identify barycenters that serves as the centroid of clusters. When finding barycenters for the source and target separately, this could be treated as the case of LOT where C z = 0 while C x , C y are defined using a squared L2 distance.…”

Section: Relationship To Ot-based Clustering Methodsmentioning

confidence: 99%

Making transport more robust and interpretable by moving data through a small number of anchor points

Lin¹,

Azabou²,

Dyer³

2020

Preprint

View full text Add to dashboard Cite

Optimal transport (OT) is a widely used technique for distribution alignment, with applications throughout the machine learning, graphics, and vision communities. Without any additional structural assumptions on transport, however, OT can be fragile to outliers or noise, especially in high dimensions. Here, we introduce a new form of structured OT that simultaneously learns low-dimensional structure in data while leveraging this structure to solve the alignment task. Compared with OT, the resulting transport plan has better structural interpretability, highlighting the connections between individual data points and local geometry, and is more robust to noise and sampling. We apply the method to synthetic as well as real datasets, where we show that our method can facilitate alignment in noisy settings and can be used to both correct and interpret domain shift.

show abstract

Section: Relationship To Ot-based Clustering Methodsmentioning

confidence: 99%

Making transport more robust and interpretable by moving data through a small number of anchor points

Lin¹,

Azabou²,

Dyer³

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…the bibliography in [42] and Section 2 below, these papers concern the discrete Schödinger bridge problem [48,34]. Hardly any attention, however, has been given to the case when only samples of continuous marginals are available (one exception is [28] which deals with using regularized optimal transport for hard and soft clustering). One might think that the latter case may be readily treated by discretizing the spatial variables through grids.…”

Section: Prior Workmentioning

confidence: 99%

“…while the Fokker-Planck equation (27b) has been replaced by the continuity equation (31b). Both Problems 6 and 7 can be thought of as regularizations of the Benamou-Brenier problem (28) and as dynamic counterparts of (9). Also notice that, precisely as in Problem (28), the optimal current velocity (29) in Problem 7 is of the gradient type.…”

Section: Stochastic Control and Fluid-dynamic Formulationsmentioning

confidence: 99%

The data-driven Schroedinger bridge

Pavon,

Tabak,

Trigila

2018

Preprint

View full text Add to dashboard Cite

Erwin Schrödinger posed, and to a large extent solved in 1931/32 the problem of finding the most likely random evolution between two continuous probability distributions. This article considers this problem in the case when only samples of the two distributions are available. A novel iterative procedure is proposed, inspired by Fortet-Sinkhorn type algorithms. Since only samples of the marginals are available, the new approach features constrained maximum likelihood estimation in place of the nonlinear boundary couplings, and importance sampling to propagate the functions ϕ and φ solving the Schrödinger system. This method is well-suited to high-dimensional settings, where introducing grids leads to numerically unfeasible or unreliable methods. The methodology is illustrated in two applications: entropic interpolation of two-dimensional Gaussian mixtures, and the estimation of integrals through a variation of importance sampling.

show abstract

“…the Euclidean distance), Wasserstein distance defines a distance between two measures as the minimal transportation cost between them. This notion of distance leads to a host of important applications, including text classification [28], clustering [23,24,14], unsupervised learning [21], semi-supervised learning [44], statistics [36,37,46,19], and others [5,39,45]. Given a set of measures in the same space, the 2-Wasserstein barycenter is defined as the measure minimizing the sum of squared 2-Wasserstein distances to all measures in the set.…”

Section: Introductionmentioning

confidence: 99%

“…The Wasserstein barycenter better captures the underlying geometric structure than the barycenter defined by the Euclidean or other distances. As a result, the Wasserstein barycenter has applications in clustering [23,24,14], image retrieval [13] and others [30,41].…”

Section: Introductionmentioning

confidence: 99%

Interior-Point Methods Strike Back: Solving the Wasserstein Barycenter Problem

Ge¹,

Wang²,

Xiong³

et al. 2019

Preprint

View full text Add to dashboard Cite

Computing the Wasserstein barycenter of a set of probability measures under the optimal transport metric can quickly become prohibitive for traditional secondorder algorithms, such as interior-point methods, as the support size of the measures increases. In this paper, we overcome the difficulty by developing a new adapted interiorpoint method that fully exploits the problem's special matrix structure to reduce the iteration complexity and speed up the Newton procedure. Different from regularization approaches, our method achieves a well-balanced tradeoff between accuracy and speed. A numerical comparison on various distributions with existing algorithms exhibits the computational advantages of our approach. Moreover, we demonstrate the practicality of our algorithm on image benchmark problems including MNIST and Fashion-MNIST.

show abstract

Parameter Estimation in Finite Mixture Models by Regularized Optimal Transport: A Unified Framework for Hard and Soft Clustering

Cited by 7 publications

References 47 publications

Making transport more robust and interpretable by moving data through a small number of anchor points

Making transport more robust and interpretable by moving data through a small number of anchor points

The data-driven Schroedinger bridge

Interior-Point Methods Strike Back: Solving the Wasserstein Barycenter Problem

Contact Info

Product

Resources

About