Risk-Based Robust Statistical Learning by Stochastic Difference-of-Convex Value-Function Optimization

Abstract

For the treatment of outliers, the paper “Risk-Based Robust Statistical Learning by Stochastic Difference-of-Convex Value-Function Optimization” by Junyi Liu and Jong-Shi Pang proposes a risk-based robust statistical learning model. Employing a variant of the conditional value-at-risk risk measure, called the interval conditional value-at-risk (In-CVaR), the model aims to exclude the risks associated with the left and right tails of the loss. The resulting nonsmooth and nonconvex model considers the population In-CVaR risk and distinguishes the upside and downside losses with asymmetric weights. For the solution of the model in both regression and classification, the authors show that the objective function is the difference of two convex functions each being the optimal objective value of a univariate convex stochastic program. A sampling and convex programming-based algorithm is developed with the appropriate control of incremental sample sizes, and its subsequential almost-sure convergence to a critical point is established. Numerical results illustrate the practical performance of the model and methodology.

Nonconvex and Nonsmooth Approaches for Affine Chance-Constrained Stochastic Programs

“…For each pair (Z N , λ), let xN,λ be a C-stationary point of (34). The result below shows that a finite λ > 0 exists such that for all λ > λ, every accumulation point of the sequence {x N,λ } is feasible for (29) and is a weak C-stationary point of this expectation-constrained problem.…”

“…Extending this basic result, a related development is the paper [51] which shows that every upper semicontinuous function is the limit of a hypo-convergent sequence of piecewise affine functions. Compared with the linear or convex random functionals considered in the existing literature of CCPs, our overall modeling framework with nonconvex and nondifferentiable random functionals together with probabilities of conjunctive and/or disjunctive random functional inequalities accommodates broader applications in operations research and statistics such as piecewise statistical estimation models [14,34] and in optimal control such as optimal path planning models [6,10].…”

“…Further numerical results with similar surrogation functions for solving related problems can be found in reference [13] for multi-composite nonconvex optimization problems arising from deep neural networks with piecewise activation functions, and in [12] for solving certain robustified nonconvex optimization problems. While the problems in these references are all deterministic, the paper [34] has some numerical results for a stochastic difference-of-convex algorithm for solving certain nonconvex risk minimization problems with expectation objectives but not the difference-ofmax-convex structure.…”