Motivated by problems arising in decentralized control problems and non-cooperative Nash games, we consider a class of strongly monotone Cartesian variational inequality (VI) problems, where the mappings either contain expectations or their evaluations are corrupted by error. Such complications are captured under the umbrella of Cartesian stochastic variational inequality problems and we consider solving such problems via stochastic approximation (SA) schemes. Specifically, we propose a scheme wherein the steplength sequence is derived by a rule that depends on problem parameters such as monotonicity and Lipschitz constants. The proposed scheme is seen to produce sequences that are guaranteed to converge almost surely to the unique solution of the problem. To cope with networked multi-agent generalizations, we provide requirements under which independently chosen steplength rules still possess desirable almostsure convergence properties. In the second part of this paper, we consider a regime where Lipschitz constants on the map are either unavailable or difficult to derive. Here, we present a local randomization technique that allows for deriving an approximation of the original mapping, which is then shown to be Lipschitz continuous with a prescribed constant. Using this technique, we introduce a locally randomized SA algorithm and provide almost sure convergence theory for the resulting sequence of iterates to an approximate solution of the original variational inequality problem. Finally, the paper concludes with some preliminary numerical results on a stochastic rate allocation problem and a stochastic Nash-Cournot game.X is a closed and convex set, D i is an open set in R ni and N i=1 n i = n. Furthermore, ξ : Ω → R d is a random variable, where Ω denotes the associated sample space and E[·] denotes the expectation with respect to ξ.Variational inequality problems assume relevance in capturing the solution sets of convex optimization and equilibrium problems [11]. Their Cartesian specializations arise from specifying the set X as a Cartesian product, i.e., X N i=1 X i . Such problems arise in the modeling of multi-agent decision-making problems such as rate allocation problems in communication networks [18,31,35], noncooperative Nash games in communication networks [1,2,39], competitive interactions in cognitive radio networks [20,29,30,38], and strategic behavior in power markets [16,17,32]. Our interest lies in regimes complicated by uncertainty, which could arise as a result of agents facing expectation-based objectives that do not have tractable analytical forms. Naturally, the Cartesian stochastic variational inequality problem framework represents an expansive model for capturing a range of such problems.Two broad avenues exist for solving such a class of problems. Of these, the first approach, referred to as the sample-average approximation (SAA) method. In adopting this approach, one uses a set of M samples {ξ 1 , . . . , ξ M } and considers the sample-average problem where an expected mapping E[Φ(x,...
