On Efficient Optimal Transport: An Analysis of Greedy and Accelerated Mirror Descent Algorithms

Abstract

We provide theoretical analyses for two algorithms that solve the regularized optimal transport (OT) problem between two discrete probability measures with at most n atoms. We show that a greedy variant of the classical Sinkhorn algorithm, known as the Greenkhorn algorithm, can be improved to O n 2 ε 2 , improving on the best known complexity bound of O n 2 ε 3 . Notably, this matches the best known complexity bound for the Sinkhorn algorithm and helps explain why the Greenkhorn algorithm can outperform the Sinkhorn algorithm in practice. Our proof technique, which is based on a primal-dual formulation and a novel upper bound for the dual solution, also leads to a new class of algorithms that we refer to as adaptive primal-dual accelerated mirror descent (APDAMD) algorithms. We prove that the complexity of these algorithms is O n 2 γ 1/2 ε , where γ > 0 refers to the inverse of the strong convexity module of Bregman divergence with respect to • ∞ . This implies that the APDAMD algorithm is faster than the Sinkhorn and Greenkhorn algorithms in terms of ε. Experimental results on synthetic and real datasets demonstrate the favorable performance of the Greenkhorn and APDAMD algorithms in practice.

Fast Algorithms for Computational Optimal Transport and Wasserstein Barycenter

“…We introduce a novel accelerated primal-dual randomized coordinate descent (APDRCD) algorithm for solving the OT problem. We provide a complexity upper bound of O( n 5/2 ε ) for the APDRCD algorithm, which is comparable to the complexity of state-of-art primal-dual algorithms for OT problems, such as the APDAGD and APDAMD algorithms [8,16]. To the best of our knowledge, this is the first accelerated primal-dual coordinate descent algorithm for computing the OT problem.…”

“…Several algorithms have been proposed to circumvent the scalability issue of the interior-point methods, including the Sinkhorn algorithm [5,12,14,24], which has a complexity bound of O( n 2 ε 2 ) where ε > 0 is the desired accuracy [8]. The Greenkhorn algorithm [1] further improves the performance of the Sinkhorn algorithm, with a theoretical complexity of O( n 2 ε 2 ) [16]. However, for large-scale applications of the OT problem, particularly in randomized and asynchronous scenarios such as computational Wasserstein barycenters, existing literature has shown that neither the Sinkhorn algorithm nor the Greenkhorn algorithm are sufficiently scalable and flexible [6,7].…”

“…We also propose a greedy version of APDRCD, which we refer to as accelerated primal-dual greedy coordinate descent (APDGCD), to further enhance practical performance. Finally, we generalize the APDRCD and APDGCD algorithms to distributed algorithms for computing the Wasserstein barycenter for multiple probability distributions.Recent research has demonstrated the advantages of the family of accelerated primal-dual algorithms [8,16] over the Sinkhorn algorithms. This family includes the adaptive primal-dual accelerated gradient descent (APDAGD) algorithm [9] and the adaptive primal-dual accelerated mirror descent (APDAMD) algorithms [16],with complexity bounds of O( n 2.5 ε ) and O( n 2 √ γ ε ), respectively, where γ ≤ √ n is the inverse of the strong complexity constant of Bregman divergence with respect to the l ∞ -norm.…”

“…Recent research has demonstrated the advantages of the family of accelerated primal-dual algorithms [8,16] over the Sinkhorn algorithms. This family includes the adaptive primal-dual accelerated gradient descent (APDAGD) algorithm [9] and the adaptive primal-dual accelerated mirror descent (APDAMD) algorithms [16],…”

Confidence Regions in Wasserstein Distributionally Robust Estimation

“…Algorithmic approaches in specific application settings include portfolio selection [5], covariance estimation [29], Kalman filtering [1], dynamic control [46,47], hypothesis testing [20] and traveling salesman problems [12]. These algorithmic advances in DRO are accompanied by significant progress in computational optimal transport (see [33,6,25] and references therein).…”