Discrete-Time Control for Systems of Interacting Objects with Unknown Random Disturbance Distributions: A Mean Field Approach

Abstract

We are concerned with stochastic control systems composed of a large number of N interacting objects sharing a common environment. The evolution of each object is determined by a stochastic difference equation where the random disturbance density ρ is unknown for the controller. We present the Markov control model (N -model) associated to the proportions of objects in each state, which is analyzed according to the mean field theory. Thus, combining convergence results as N → ∞ (the mean field limit) with a suitable statistical estimation method for ρ, we construct the so-named eventually asymptotically optimal policies for the N -model under a discounted optimality criterion. A consumption-investment problem is analyzed to illustrate our results.

Dynamic performances of a bird-like flapping wing robot under randomly uncertain disturbances

“…A linear time invariant system was also used to generate the first-order Markov colored noise to model the small random disturbances in [8]. C. G. Higuera-Chan et al described the random disturbances with stochastic difference equations, and assumed the disturbances were observable, independent, and uniformly distributed with a density ρ [9]. The randomly uncertain disturbances were regarded as bounded noise processes that were imposed on a system when exploring noise-induced complicated dynamics [10].…”

Control systems of interacting objects modeled as a game against nature under a mean field approach

“…Within the context (1)-(2), the specific case where {ξ n t } is a family of observable, independent and identically distributed random variables with common unknown distribution µ, i.e. µ t = µ for all t ≥ 0, was previously analyzed in [13]. In particular, it was assumed that the distribution µ has an unknown density ρ, so it was possible to implement statistical density estimation schemes for ρ together with control procedures, to prove existence of nearly optimal policies.…”

“…To the best of our knowledge, our approach has not been previously studied in the literature and its novelty relies in mixing the aforementioned two techniques. Furthermore, this work can be considered as an extension of [13] in the sense that our results are applicable to difference-equation models as (1)-(2), but assuming that the driving process {ξ n t } is nonobservable with unknown distribution. The paper is organized as follows.…”

Mean Field Markov Decision Processes

“…Finite as well as infinite-horizon discounted reward problems are considered. In [18] the authors also investigate convergence in a discounted reward problem, however consider the situation that the random disturbance density in unknown. A consumption-investment example is discussed there.…”