Data-driven Control of Agent-based Models: an Equation/Variable-free Machine Learning Approach

Abstract

We present an Equation/Variable free machine learning (EVFML) framework for the control of the collective dynamics of complex/multiscale systems modelled via microscopic/agent-based simulators. The approach obviates the need for construction of surrogate, reduced-order models. The proposed implementation consists of three steps: (A) from high-dimensional agent-based simulations, machine learning (in particular, non-linear manifold learning (Diffusion Maps (DMs)) helps identify a set of coarse-grained variables that parametrize the low-dimensional manifold on which the emergent/collective dynamics evolve. The out-of-sample extension and pre-image problems, i.e. the construction of non-linear mappings from the high-dimensional input space to the low-dimensional manifold and back, are solved by coupling DMs with the Nyström extension and Geometric Harmonics, respectively; (B) having identified the manifold and its coordinates, we exploit the Equation-free approach to perform numerical bifurcation analysis of the emergent dynamics; then (C) based on the previous steps, we design data-driven embedded wash-out controllers that drive the agent-based simulators to their intrinsic, imprecisely known, emergent open-loop unstable steady-states, thus demonstrating that the scheme is robust against numerical approximation errors and modelling uncertainty. The efficiency of the framework is illustrated by controlling emergent unstable (i) traveling waves of a deterministic agent-based model of traffic dynamics, and (ii) equilibria of a stochastic financial market agent model with mimesis.

Continuification Control of Large-Scale Multiagent Systems in a Ring

“…Macroscopic descriptions were also used in some multiagent reinforcement learning scenarios [16], [17]. Other methodologies recently proposed in the literature are based on the use of graphons [18] and data and manifold learning [19], [20].…”

Mean-field models of dynamics on networks via moment closure: An automated procedure

“…Sometimes one can find an appropriate moment space and closure by taking account of the characteristic features of the process in consideration, leading to a description with greater efficiency compared to what is obtainable by using the size-based moment space and closing via independence assumptions [59]. We expect that for finding good moment spaces and closures, equationfree and machine-learning methods [60,61] will play an important role, in particular because the most appropriate low-dimensional descriptions or their closures may not necessarily be available in closed form [5,6,60,61]. It is subject of future work to explore if and how these different approaches relate.…”