Gaussian-Smooth Optimal Transport: Metric Structure and Statistical Efficiency

Abstract

Optimal transport (OT), and in particular the Wasserstein distance, has seen a surge of interest and applications in machine learning. However, empirical approximation under Wasserstein distances suffers from a severe curse of dimensionality, rendering them impractical in high dimensions. As a result, entropically regularized OT has become a popular workaround. However, while it enjoys fast algorithms and better statistical properties, it looses the metric structure that Wasserstein distances enjoy. This work proposes a novel Gaussian-smoothed OT (GOT) framework, that achieves the best of both worlds: preserving the 1-Wasserstein metric structure while alleviating the empirical approximation curse of dimensionality. Furthermore, as the Gaussian-smoothing parameter shrinks to zero, GOT Γ-converges towards classic OT (with convergence of optimizers), thus serving as a natural extension. An empirical study that supports the theoretical results is provided, promoting Gaussian-smoothed OT as a powerful alternative to entropic OT.1 Any p-Wasserstein distance has these properties. 2 Note that while Wasserstein-type GANs in practice typically use the two-sample setup since the generator distribution is intractable to compute, fundamentally the GAN actually corresponds to a one-sample setup since infinite samples can be obtained from the generator network.

Asymptotics of smoothed Wasserstein distances

“…Goldfeld et al [16] extended this analysis to the total variation distance, relative entropy, and 2-Wasserstein distance, as well as to distributions with unbounded support. Motivated by these findings, Goldfeld and Greenewald [15] propose to study this smoothed Wasserstein distance as a statistically attractive variant of the standard Wasserstein distances.…”