Model predictive complex system control from observational and interventional data

Abstract

Complex systems, characterized by intricate interactions among numerous entities, give rise to emergent behaviors whose data-driven modeling and control are of utmost significance, especially when there is abundant observational data but the intervention cost is high. Traditional methods rely on precise dynamical models or require extensive intervention data, often falling short in real-world applications. To bridge this gap, we consider a specific setting of the complex systems control problem: how to control complex systems through a few online interactions on some intervenable nodes when abundant observational data from natural evolution is available. We introduce a two-stage model predictive complex system control framework, comprising an offline pre-training phase that leverages rich observational data to capture spontaneous evolutionary dynamics and an online fine-tuning phase that uses a variant of model predictive control to implement intervention actions. To address the high-dimensional nature of the state-action space in complex systems, we propose a novel approach employing action-extended graph neural networks to model the Markov decision process of complex systems and design a hierarchical action space for learning intervention actions. This approach performs well in three complex system control environments: Boids, Kuramoto, and Susceptible-Infectious-Susceptible (SIS) metapopulation. It offers accelerated convergence, robust generalization, and reduced intervention costs compared to the baseline algorithm. This work provides valuable insights into controlling complex systems with high-dimensional state-action spaces and limited intervention data, presenting promising applications for real-world challenges.