Several Reinforcement Learning Methods in Mean-Field Games with Binary Action Spaces

Abstract

Recent years have witnessed significant progress in the sub-field of machine learning known as reinforcement learning, in which interactions between intelligent agents and the environment enable agents to learn and solve sequential decision-making problems through accumulating rewards with delays. Despite much success in single-player settings, reinforcement learning in multi-agent domains remains a challenging task in many aspects. In this thesis, the mean-field approach will be used to study binary action space stochastic games with a sufficiently large number of players that can be generalized to the multi-population case. Based on the mean-field approximation, several algorithms will be implemented and compared in numerical experiments to visualize their convergence to the equilibrium policy.It is my greatest honor to dedicate this thesis to my supervisor, Minyi Huang, without whom by no means could I finish this thesis. It was Professor Huang who introduced me to the mean-field games theory; it was Professor Huang who academically guided me and financially supported me; it was Professor Huang who cheered me up when I was deeply frustrated by the slow progress of my work. During the pandemic, face-to-face communication was unavailable, but Professor Huang strived to maintain a routine video communication with me. Moreover, Professor Huang invited me to a variety of virtual academic conferences, which facilitated me to grasp a deeper understanding of the meanfield game theory and reinforcement learning.I would like to also dedicate my thesis to my parents. Due to Covid-19, I had to spend another academic year to complete my thesis, and my parents were still willing to support me, both financially and emotionally. My family originally comes from Wuhan, where the first outbreak of Covid-19 was observed, so last year was a tough year for my entire family. I hope this thesis marks a new beginning for me and my family.