Ambiguity set and learning via Bregman and Wasserstein

Abstract

Construction of ambiguity set in robust optimization relies on the choice of divergences between probability distributions. In distribution learning, choosing appropriate probability distributions based on observed data is critical for approximating the true distribution. To improve the performance of machine learning models, there has recently been interest in designing objective functions based on Lp-Wasserstein distance rather than the classical Kullback-Leibler (KL) divergence. In this paper, we derive concentration and asymptotic results using Bregman divergence. We propose a novel asymmetric statistical divergence called Wasserstein-Bregman divergence as a generalization of L2-Wasserstein distance. We discuss how these results can be applied to the construction of ambiguity set in robust optimization.

Transport information Bregman divergences

“…Remark 3 (Comparisons with Wasserstein-Bregman divergence). We notice that formulation (i) is similar but different with divergence functionals defined in [13,33]. In our formulation, joint density π is the optimal one from the L 2 -Wasserstein space; while in [13] or [33], joint density π solves the related linear programming problem w.r.t.…”

“…There are joint works in the literature between optimal transport and information geometry to study Bregman divergences [2,7,13,22,23,25,33,34]. In particular, [13,33,34] apply linear programming formulations of optimal transport.…”

“…There are joint works in the literature between optimal transport and information geometry to study Bregman divergences [2,7,13,22,23,25,33,34]. In particular, [13,33,34] apply linear programming formulations of optimal transport. They use divergence functions on sample space as ground costs to construct the ones in probability density space.…”

“…In this space, the L 2 -Wasserstein distance shows a particular convexity property towards mapping functions [3,24]. This convexity property nowadays has vast applications in fluid dynamics [11,16,17], inverse problems [35], and AI inference problems [2,13,31]. Natural questions arise.…”

Regularized Inverse Reinforcement Learning

“…Note that for two continuous distributions P 1 and P 2 having probability density functions p 1 (x) and p 2 (x), respectively, the Bregman divergence can be defined as (Guo et al, 2017;Jones & Byrne, 1990)…”