A Wasserstein-Type Distance in the Space of Gaussian Mixture Models

Delon, Julie; Desolneux, Agnès

doi:10.1137/19m1301047

Cited by 58 publications

(52 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Next, we study the practical convergence of projected stochastic gradient descent (Algorithm 3). Using the fact that the true Wasserstein barycenter of one-dimensional Gaussian measures has closed-form expression for the mean and the variance [22], we study the convergence to the true barycenter of the generated truncated Gaussian measures. Figure 1 illustrates the convergence in the 2-Wasserstein distance within 40 s.…”

Section: The Total Complexity Of Algorithm 3 Ismentioning

confidence: 99%

Stochastic approximation versus sample average approximation for Wasserstein barycenters

Dvinskikh

2021

Optimization Methods and Software

View full text Add to dashboard Cite

In the machine learning and optimization community, there are two main approaches for the convex risk minimization problem, namely the Stochastic Approximation (SA) and the Sample Average Approximation (SAA). In terms of the oracle complexity (required number of stochastic gradient evaluations), both approaches are considered equivalent on average (up to a logarithmic factor). The total complexity depends on a specific problem, however, starting from the work [A. Nemirovski, A. Juditsky, G. Lan, and A. Shapiro, Robust stochastic approximation approach to stochastic programming, SIAM. J. Opt. 19 (2009), pp. 1574-1609] it was generally accepted that the SA is better than the SAA. We show that for the Wasserstein barycenter problem, this superiority can be inverted. We provide a detailed comparison by stating the complexity bounds for the SA and SAA implementations calculating barycenters defined with respect to optimal transport distances and entropy-regularized optimal transport distances. As a byproduct, we also construct confidence intervals for the barycenter defined with respect to entropyregularized optimal transport distances in the 2 -norm. The preliminary results are derived for a general convex optimization problem given by the expectation to have other applications besides the Wasserstein barycenter problem.

show abstract

Section: The Total Complexity Of Algorithm 3 Ismentioning

confidence: 99%

Stochastic approximation versus sample average approximation for Wasserstein barycenters

Dvinskikh

2021

Optimization Methods and Software

View full text Add to dashboard Cite

show abstract

“…So, it is worth exploring noise distributions other than the exponential mechanism. For P X|S (•|s, ρ) being Gaussian distribution or Gaussian mixture model for all s, the Kantorovich optimal transport plan π * is fully characterized by the mean and covariance matrix (Takatsu, 2010;Delon and Desolneux, 2020). Since Gaussian models are widely used in machine learning, it is of interest if the Kantorovich mechanism can be apply to the privacy-preserving pattern recognition problems.…”

Section: Discussionmentioning

confidence: 99%

Kantorovich Mechanism for Pufferfish Privacy

Ding¹

2022

Preprint

View full text Add to dashboard Cite

Pufferfishprivacy achieves ǫindistinguishability over a set of secret pairs in the disclosed dataset. This paper studies how to attain pufferfish privacy by the exponential mechanism, an additive noise scheme that generalizes Gaussian and Laplace noise.A sufficient condition is derived showing that pufferfish privacy is attained by calibrating noise to the sensitivity of the Kantorovich optimal transport plan. Such a plan can be directly computed by using the data statistics conditioned on the secret, the prior knowledge about the system. It is shown that Gaussian noise provides better data utility than Laplace noise when the privacy budget ǫ is small. The sufficient condition is then relaxed to reduce the noise power. Experimental results show that the relaxed sufficient condition improves data utility of the pufferfish private data regulation schemes.

show abstract

“…We also provide further justification why the Wasserstein Distance is a sensible metric to use. It is well known that a mixture of Gaussians can converge in distribution to any continuous random variable, however existing work has shown that a mixture of Gaussians can approximate any discrete distribution in the Wasserstein Distance arbitrarily well [20].…”

Section: Wasserstein Distancementioning

confidence: 99%

Continualization of Probabilistic Programs With Correction

Laurel

Misailovíc

2020

Programming Languages and Systems

View full text Add to dashboard Cite

Probabilistic Programming offers a concise way to represent stochastic models and perform automated statistical inference. However, many real-world models have discrete or hybrid discrete-continuous distributions, for which existing tools may suffer non-trivial limitations. Inference and parameter estimation can be exceedingly slow for these models because many inference algorithms compute results faster (or exclusively) when the distributions being inferred are continuous. To address this discrepancy, this paper presents Leios. Leios is the first approach for systematically approximating arbitrary probabilistic programs that have discrete, or hybrid discrete-continuous random variables. The approximate programs have all their variables fully continualized. We show that once we have the fully continuous approximate program, we can perform inference and parameter estimation faster by exploiting the existing support that many languages offer for continuous distributions. Furthermore, we show that the estimates obtained when performing inference and parameter estimation on the continuous approximation are still comparably close to both the true parameter values and the estimates obtained when performing inference on the original model.

show abstract

A Wasserstein-Type Distance in the Space of Gaussian Mixture Models

Cited by 58 publications

References 30 publications

Stochastic approximation versus sample average approximation for Wasserstein barycenters

Stochastic approximation versus sample average approximation for Wasserstein barycenters

Kantorovich Mechanism for Pufferfish Privacy

Continualization of Probabilistic Programs With Correction

Contact Info

Product

Resources

About