Deep Reinforcement Learning for Online Control of Stochastic Partial Differential Equations

Abstract

In many areas, such as the physical sciences, life sciences, and finance, control approaches are used to achieve a desired goal in complex dynamical systems governed by differential equations. In this work we formulate the problem of controlling stochastic partial differential equations (SPDE) as a reinforcement learning problem. We present a learning-based, distributed control approach for online control of a system of SPDEs with high dimensional state-action space using deep deterministic policy gradient method. We tested the performance of our method on the problem of controlling the stochastic Burgers' equation, describing a turbulent fluid flow in an infinitely large domain.

Dynamic Noises of Multi-Agent Environments Can Improve Generalization: Agent-based Models meets Reinforcement Learning

“…The dynamics of the latter are out of reach for mathematical methods (Epstein & Axtell (1996); Colman (1998)), i.e., they cannot be explicitly governed by ordinary differential equations (ODE) or partial differential equations (PDE). This is in contrast to many real-word applications such as robotics (Kober et al (2013)), ventilating (Farahmand et al (2016)), and fluid dynamics Pirmorad et al (2021) whose complex dynamical systems can be captured using ODEs and PDEs. Moreover, it has been shown that RL agents are prone to exploiting idiosyncrasies of specific implementations of RL environments from which they learn infeasible behaviors in the real world (Heess et al (2017)).…”