Structure Learning for Approximate Solution of Many-Player Games

Abstract

Games with many players are difficult to solve or even specify without adopting structural assumptions that enable representation in compact form. Such structure is generally not given and will not hold exactly for particular games of interest. We introduce an iterative structure-learning approach to search for approximate solutions of many-player games, assuming only black-box simulation access to noisy payoff samples. Our first algorithm, K-Roles, exploits symmetry by learning a role assignment for players of the game through unsupervised learning (clustering) methods. Our second algorithm, G3L, seeks sparsity by greedy search over local interactions to learn a graphical game model. Both algorithms use supervised learning (regression) to fit payoff values to the learned structures, in compact representations that facilitate equilibrium calculation. We experimentally demonstrate the efficacy of both methods in reaching quality solutions and uncovering hidden structure, on both perfectly and approximately structured game instances.

Navigating the landscape of multiplayer games

“…However, any other general-sum payoff parameterization approaches (e.g., even direct generation of the payoff entries) can also be used to avoid the zero-sum constraint. The study of normal-form games continues to play a prominent role in the game theory and machine learning literature [102][103][104][105][106][107][108] and as such the procedural generation of normal-form games can play an important role in the research community. An important line of future work will involve investigating means of generalizing this approach to generation of more complex classes of games.…”

Towards multi‐agent reinforcement learning‐driven over‐the‐counter market simulations

“…We now proceed to making assumptions on the rewards and state transition kernel that we call type‐symmetry , since they are similar to the anonymity/role‐symmetry assumption in Li and Wellman (2020). Their only purpose is to guarantee that the expected reward of an agent in Equation (2) only depends on its supertype $$\Lambda _i^\kappa$$.…”

Evolution Strategies for Approximate Solution of Bayesian Games

“…Model Learning However in our problem, the issue for replicator dynamics or other Nash-solvers is that we do not have the exact evaluation of deviation payoff u(s, σ) but only its stochastic query values, which may require a significant number of samples to control the variance for each update, and inhibit Nash-solvers from converging to stable solutions. To reduce sample complexity and computational intractability, we adopt a supervised model-learning approach (Vorobeychik, Wellman, and Singh 2007;Wiedenbeck, Yang, and Wellman 2018;Sokota, Ho, and Wiedenbeck 2019;Li and Wellman 2020) that regresses the pure-strategy payoff function of this finite restricted game model, and further provides RD with deviation estimation for mixture computation.…”