Distributionally Robust Inverse Covariance Estimation: The Wasserstein Shrinkage Estimator

Nguyen, Viet Anh; Kühn, Daniel; Esfahani, Peyman Mohajerin

doi:10.48550/arxiv.1805.07194

Cited by 10 publications

(17 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The second equality holds because of strong duality [38]. With (23), we can reach the conclusion of Lemma 1.…”

Section: Wasserstein Robust Graph Learning With Gaussian Prior Assump...mentioning

confidence: 53%

“…Proof: The proof is similar with Proposition 4.3 in [38]. Since g(γ, L) is a multivariable function with γ and L jointly, we calculate the Hessian matrix of g(γ, L) and prove the positive definite of it.…”

Section: B Solving the Convex Optimizationmentioning

confidence: 85%

“…Proof: From Theorem 1, the origin inf-sup problem has been proven to be tantamount to an inf-problem (24). The remaining proof is similar with Corollary 2.9 in [38]. Specifically, in (24), we focus our attention on the nonlinear term φ(γ, L)…”

Section: Reformulation As Semi-definite Programming (Sdp)mentioning

confidence: 93%

“…Obviously, expectation operator in (11) is defined on the probability of infinite dimensions, and it is difficult to solve such an optimization problems directly. For the same reason as we present in Lemma 1, the following dual form of ( 38) is obtained by using the definition of W α (P, P n ), Lemma 4: For any γ > 0, the dual form of the worst case risk in (38) can be written as:…”

Section: Wasserstein Robust Graph Learning Without Prior Assumption A...mentioning

confidence: 99%

“…In fact, φ(γ, L) is a matrix fractional function described in [44] and has the following reformulation [38]:…”

Section: Reformulation As Semi-definite Programming (Sdp)mentioning

confidence: 99%

See 4 more Smart Citations

Robust Graph Learning Under Wasserstein Uncertainty

Zhang,

Xu,

Liu

et al. 2021

Preprint

View full text Add to dashboard Cite

Graphs are playing a crucial role in different fields since they are powerful tools to unveil intrinsic relationships among signals. In many scenarios, an accurate graph structure representing signals is not available at all and that motivates people to learn a reliable graph structure directly from observed signals. However, in real life, it is inevitable that there exists uncertainty in the observed signals due to noise measurements or limited observability, which causes a reduction in reliability of the learned graph. To this end, we propose a graph learning framework using Wasserstein distributionally robust optimization (WDRO) which handles uncertainty in data by defining an uncertainty set on distributions of the observed data. Specifically, two models are developed, one of which assumes all distributions in uncertainty set are Gaussian distributions and the other one has no prior distributional assumption. Instead of using interior point method directly, we propose two algorithms to solve the corresponding models and show that our algorithms are more time-saving. In addition, we also reformulate both two models into Semi-Definite Programming (SDP), and illustrate that they are intractable in the scenario of large-scale graph. Experiments on both synthetic and real world data are carried out to validate the proposed framework, which show that our scheme can learn a reliable graph in the context of uncertainty.

show abstract

“…The second equality holds because of strong duality [38]. With (23), we can reach the conclusion of Lemma 1.…”

Section: Wasserstein Robust Graph Learning With Gaussian Prior Assump...mentioning

confidence: 53%

Section: B Solving the Convex Optimizationmentioning

confidence: 85%

Section: Reformulation As Semi-definite Programming (Sdp)mentioning

confidence: 93%

Section: Wasserstein Robust Graph Learning Without Prior Assumption A...mentioning

confidence: 99%

“…In fact, φ(γ, L) is a matrix fractional function described in [44] and has the following reformulation [38]:…”

Section: Reformulation As Semi-definite Programming (Sdp)mentioning

confidence: 99%

See 3 more Smart Citations

Robust Graph Learning Under Wasserstein Uncertainty

Zhang,

Xu,

Liu

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

Wasserstein Distributionally Robust Optimization: Theory and Applications in Machine Learning

Kühn

Esfahani

Nguyên

et al. 2019

Operations Research &Amp; Management Science in the Age of Analytics

Self Cite

245

216

View full text Add to dashboard Cite

Many decision problems in science, engineering and economics are affected by uncertain parameters whose distribution is only indirectly observable through samples. The goal of data-driven decision-making is to learn a decision from finitely many training samples that will perform well on unseen test samples. This learning task is difficult even if all training and test samples are drawn from the same distributionespecially if the dimension of the uncertainty is large relative to the training sample size. Wasserstein distributionally robust optimization seeks data-driven decisions that perform well under the most adverse distribution within a certain Wasserstein distance from a nominal distribution constructed from the training samples. In this tutorial we will argue that this approach has many conceptual and computational benefits. Most prominently, the optimal decisions can often be computed by solving tractable convex optimization problems, and they enjoy rigorous out-of-sample and asymptotic consistency guarantees. We will also show that Wasserstein distributionally robust optimization has interesting ramifications for statistical learning and motivates new approaches for fundamental learning tasks such as classification, regression, maximum likelihood estimation or minimum mean square error estimation, among others.

show abstract

Distributionally Robust Learning

Chen

Paschalidis

2020

FNT in Optimization

View full text Add to dashboard Cite

This monograph develops a comprehensive statistical learning framework that is robust to (distributional) perturbations in the data using Distributionally Robust Optimization (DRO) under the Wasserstein metric. Beginning with fundamental properties of the Wasserstein metric and the DRO formulation, we explore duality to arrive at tractable formulations and develop finite-sample, as well as asymptotic, performance guarantees. We consider a series of learning problems, including (i) distributionally robust linear regression; (ii) distributionally robust regression with group structure in the predictors; (iii) distributionally robust multi-output regression and multiclass classification, (iv) optimal decision making that combines distributionally robust regression with nearest-neighbor estimation; (v) distributionally robust semi-supervised learning, and (vi) distributionally robust reinforcement learning. A tractable DRO relaxation for each problem is being derived, establishing a connection between robustness and regularization, and obtaining bounds on the prediction and estimation errors of the solution. Beyond theory, we include numerical experiments and case studies using synthetic and real data. The real data experiments are all associated with various health informatics problems, an application area which provided the initial impetus for this work.

show abstract

Distributionally Robust Inverse Covariance Estimation: The Wasserstein Shrinkage Estimator

Cited by 10 publications

References 0 publications

Robust Graph Learning Under Wasserstein Uncertainty

Robust Graph Learning Under Wasserstein Uncertainty

Wasserstein Distributionally Robust Optimization: Theory and Applications in Machine Learning

Distributionally Robust Learning

Contact Info

Product

Resources

About