Optimal Solutions to Infinite-Player Stochastic Teams and Mean-Field Teams

Abstract

We study stochastic static teams with countably infinite number of decision makers, with the goal of obtaining (globally) optimal policies under a decentralized information structure. We present sufficient conditions to connect the concepts of team optimality and person by person optimality for static teams with countably infinite number of decision makers. We show that under uniform integrability and uniform convergence conditions, an optimal policy for static teams with countably infinite number of decision makers can be established as the limit of sequences of optimal policies for static teams with N decision makers as N → ∞. Under the presence of a symmetry condition, we relax the conditions and this leads to optimality results for a large class of mean-field optimal team problems where the existing results have been limited to personby-person-optimality and not global optimality (under strict decentralization). In particular, we establish the optimality of symmetric (i.e., identical) policies for such problems. As a further condition, this optimality result leads to an existence result for mean-field teams. We consider a number of illustrative examples where the theory is applied to setups with either infinitely many decision makers or an infinite-horizon stochastic control problem reduced to a static team.

Convex Symmetric Stochastic Dynamic Teams and Their Mean-Field Limit

“…REMARK 1. The concepts of exchangeable and symmetrically optimal dynamic teams in this paper are generalizations of those for static teams in [35,36]. However, here, the value of the cost function may not be invariant under exchanging γ i t with γ j k for k = t, k, t = 0, .…”

“…For any number of DMs, this symmetry structure is more relaxed when compared with the symmetry results developed earlier, e.g. in [35,36] which focused on problems under a static information structure, and is applicable for dynamic teams which may not admit a static reduction, as long as convexity in policies holds for the team problem.…”

“…On the other hand, in our paper since we work under the convexity assumption, the information structure does not allow for mean-field coupling in the dynamics. We also note that in prior work, [35], we studied static teams where under convexity and more restrictive symmetry conditions, global optimality of a limit policy of a sequence of N -DM optimal policies has been established.…”

“…(i) We define the notion of exchangeable teams and symmetric information structures, and we show that, for convex exchangeable dynamic teams with finite horizons, optimal policies exhibit a symmetry structure (Theorem 2.7). For any number of DMs, this symmetry structure is more relaxed when compared with the symmetry results developed in [35,36] and is applicable for dynamic teams which may not admit a static reduction, as long as convexity in policies holds for the team problem. (ii) For convex mean-field teams with a symmetric information structure, through concentration of measure arguments, we establish the convergence of optimal polices for mean-field teams with N decision makers to the corresponding optimal policies for mean-field teams (see Theorem 3.2).…”

“…Optimality of symmetric policies for convex dynamic teams with a symmetric information structure. In the following, we define notions of exchangeable and symmetrically optimal teams analogous to [35,36] for dynamic teams.…”