A regularized adaptive steplength stochastic approximation scheme for monotone stochastic variational inequalities

Abstract

Motivated by problems arising in decentralized control problems and non-cooperative Nash games, we consider a class of strongly monotone Cartesian variational inequality (VI) problems, where the mappings either contain expectations or their evaluations are corrupted by error. Such complications are captured under the umbrella of Cartesian stochastic variational inequality problems and we consider solving such problems via stochastic approximation (SA) schemes. Specifically, we propose a scheme wherein the steplength sequence is derived by a rule that depends on problem parameters such as monotonicity and Lipschitz constants. The proposed scheme is seen to produce sequences that are guaranteed to converge almost surely to the unique solution of the problem. To cope with networked multi-agent generalizations, we provide requirements under which independently chosen steplength rules still possess desirable almostsure convergence properties. In the second part of this paper, we consider a regime where Lipschitz constants on the map are either unavailable or difficult to derive. Here, we present a local randomization technique that allows for deriving an approximation of the original mapping, which is then shown to be Lipschitz continuous with a prescribed constant. Using this technique, we introduce a locally randomized SA algorithm and provide almost sure convergence theory for the resulting sequence of iterates to an approximate solution of the original variational inequality problem. Finally, the paper concludes with some preliminary numerical results on a stochastic rate allocation problem and a stochastic Nash-Cournot game.X is a closed and convex set, D i is an open set in R ni and N i=1 n i = n. Furthermore, ξ : Ω → R d is a random variable, where Ω denotes the associated sample space and E[·] denotes the expectation with respect to ξ.Variational inequality problems assume relevance in capturing the solution sets of convex optimization and equilibrium problems [11]. Their Cartesian specializations arise from specifying the set X as a Cartesian product, i.e., X N i=1 X i . Such problems arise in the modeling of multi-agent decision-making problems such as rate allocation problems in communication networks [18,31,35], noncooperative Nash games in communication networks [1,2,39], competitive interactions in cognitive radio networks [20,29,30,38], and strategic behavior in power markets [16,17,32]. Our interest lies in regimes complicated by uncertainty, which could arise as a result of agents facing expectation-based objectives that do not have tractable analytical forms. Naturally, the Cartesian stochastic variational inequality problem framework represents an expansive model for capturing a range of such problems.Two broad avenues exist for solving such a class of problems. Of these, the first approach, referred to as the sample-average approximation (SAA) method. In adopting this approach, one uses a set of M samples {ξ 1 , . . . , ξ M } and considers the sample-average problem where an expected mapping E[Φ(x,...

Stochastic Variational Inequality Problems: Applications, Analysis, and Algorithms

“…A key shortcoming of standard stochastic approximation schemes is the relatively ad hoc nature of the choice of step-length sequences. Yousefian et al [116,117] developed distributed stochastic approximation schemes where users can independently choose a step-length rule. Importantly, these rules collectively allow for minimizing a suitably defined error bound and are equipped with almost-sure convergence guarantees.…”

“…Although, this choice allows for almost-sure convergence, it has been observed that such choices may lead to poor performance in practice. Motivated by this shortcoming, Yousefian et al [117,116] developed step-length rules that allow for prescribing step-length sequences in accordance with problem parameters. More specifically, such a rule derived a sequence in a convex stochastic optimization regime by leveraging three parameters: (i) Lipschitz constant of the gradients, (ii) strong convexity constant, and (ii) diameter of the set X.…”

A distributed adaptive steplength stochastic approximation method for monotone stochastic Nash Games

“…It can be shown that the mapping F is strongly monotone and Lipschitz with specified parameters (cf. [19]). We solve the bandwidth-sharing problem for 12 different settings of parameters shown in Table I.…”

“…Lizarraga et al [14] considered a family of two person Mutil-Plant game and developed Stackelberg-Nash equilibrium conditions based on the Robust Maximum Principle. More recently, Yousefian et al [1], [15] developed centralized adaptive stepsize SA schemes for solving stochastic optimization problems and variational inequalities. The main contribution of the current paper lies in developing a class of distributed adaptive stepsize rules for SA scheme in which each agent chooses its own stepsizes without any specific information about other agents stepsize policy.…”

The pseudomonotone stochastic variational inequality problem: Analytical statements and stochastic extragradient schemes

“…The second approach, inspired by the seminal work by Robbins and Monro [10], is that of stochastic approximation and there has been a surge of recent effort applying such techniques to stochastic variational inequality problems [11], [12], [13], [14]. While most of the above contributions rely on standard diminishing steplength sequences, Yousefian and his coauthors [15], [16] propose techniques in which the steplength sequences are tuned to problem parameters and minimize a suitably defined meansquared error bound. Yet, almost of these statements are limited by their ability to accommodate strongly, strictly, or merely monotone maps.…”