Solving Heterogeneous General Equilibrium Economic Models with Deep Reinforcement Learning

Abstract

General equilibrium macroeconomic models are a core tool used by policymakers to understand a nation's economy. They represent the economy as a collection of forward-looking actors whose behaviours combine, possibly with stochastic effects, to determine global variables (such as prices) in a dynamic equilibrium. However, standard semi-analytical techniques for solving these models make it difficult to include the important effects of heterogeneous economic actors. The COVID-19 pandemic has further highlighted the importance of heterogeneity, for example in age and sector of employment, in macroeconomic outcomes and the need for models that can more easily incorporate it. We use techniques from reinforcement learning to solve such models incorporating heterogeneous agents in a way that is simple, extensible, and computationally efficient. We demonstrate the method's accuracy and stability on a toy problem for which there is a known analytical solution, its versatility by solving a general equilibrium problem that includes global stochasticity, and its flexibility by solving a combined macroeconomic and epidemiological model to explore the economic and health implications of a pandemic. The latter successfully captures plausible economic behaviours induced by differential health risks by age.

A computational neuroscience perspective on subjective wellbeing within the active inference framework

“…A branch of game theory called mechanism design, which can be thought of as the "engineering" side of economic theory, is one appropriate mathematical approach for identifying policies in this manner (Maskin, 2008). Based on the present discussion, future work might consider mechanism design using agents that are themselves engaged in active inference (e.g., within a general equilibrium macroeconomic model that is being utilized to understand a country's SWB (Hill et al, 2021)). Parallel work in the RL literature has developed similar large-scale simulations to develop dynamic taxation and subsidy policies that consider multiple objectives, policy levers, and behavioral responses from strategic actors that optimize for their individual objectives (Trott et al, 2021).…”

Deep Reinforcement Learning: Emerging Trends in Macroeconomics and Future Prospects

“…First, use of the algorithms as a solution method to find optimal policy or policy response function, such as Hinterlang and Tänzer (2022) and Covarrubias (2022). This area also extends to solving general equilibrium models, such as Chen et al (2021), Hill et al (2021), and Curry et al (2022). Moreover, Chen et al (2021) also study the learnability of rational expectation solutions in a general equilibrium model with multiple equilibria.…”

“…Chen et al (2021), Shi (2021aShi ( , 2021b all study learning in the case of a single representative household. Hill et al (2021) and Curry et al (2022) look into cases of general equilibrium models with multiple learning agent. Hill et al (2021) show how to solve three rational expectations equilibrium models with discrete heterogenous agents instead of a continuum of agents or a single representative agent.…”

“…Hill et al (2021) and Curry et al (2022) look into cases of general equilibrium models with multiple learning agent. Hill et al (2021) show how to solve three rational expectations equilibrium models with discrete heterogenous agents instead of a continuum of agents or a single representative agent. These models are precautionary saving model; the interaction between a pandemic and the macroeconomy (an 'epi-macro' model), with stochasticity in health statuses; and a macroeconomic model which has global stochasticity, i.e., where the background is changing in a way that the agents are unable to predict.…”

Generative Adversarial Equilibrium Solvers

“…In regard to finding optimal economic designs, deep learning has been used for problems of auction design (Dütting et al, 2019;Curry et al, 2022c;Tacchetti et al, 2019;Curry et al, 2022a;Gemp et al, 2022;Rahme et al, 2020) and matching (Ravindranath et al, 2021). In regard to solving for equilibria, some recent works have tried to solve for Nash equilibria in auctions (Heidekrüger et al, 2019;Bichler et al, 2021), and dynamic stochastic general equilibrium models (Curry et al, 2022b;Chen et al, 2021;Hill et al, 2021).…”