Inverse Game Theory: Learning Utilities in Succinct Games

Kuleshov, Volodymyr; Schrijvers, Okke

doi:10.1007/978-3-662-48995-6_30

Cited by 24 publications

(20 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Under the condition (I −P iP + i )p i = 0 and by noting thatP i is positive semidefinite due to P i (0) being positive semidefinite, [33,Proposition 15.2] gives that this quadratic program is solved by anyα i satisfying (16) for any b ∈ R Mi+n−1−rP i . The first theorem assertion (15) and (16) follows. Now, from (16), clearly,…”

Section: B Solution Of Soft-constrained Methodsmentioning

confidence: 83%

Inverse Open-Loop Noncooperative Differential Games and Inverse Optimal Control

Molloy

Inga

Flad

et al. 2020

IEEE Trans. Automat. Contr.

View full text Add to dashboard Cite

We consider the problem of computing parameters of player cost functionals such that given state and control trajectories constitute an open-loop Nash equilibrium for a noncooperative differential game. We propose two methods for solving this inverse differential game problem and novel conditions under which our methods compute unique cost-functional parameters. Our conditions are analogous to persistence of excitation conditions in adaptive control and parameter estimation. The efficacy of our methods is illustrated in simulations. Index Terms-Game theory, inverse differential games, inverse optimal control, optimal control.

show abstract

Section: B Solution Of Soft-constrained Methodsmentioning

confidence: 83%

Inverse Open-Loop Noncooperative Differential Games and Inverse Optimal Control

Molloy

Inga

Flad

et al. 2020

IEEE Trans. Automat. Contr.

View full text Add to dashboard Cite

show abstract

“…Approx. IESAR (ours) Markov Nash Yes MA-AIRL (Yu et al, 2019) Markov LSBRE Yes (Lin et al, 2019) Markov Various No (Natarajan et al, 2010) Markov Cooperative Game No (Waugh et al, 2011) Normal-Form Correlated Yes (Bestick et al, 2013) Normal-Form Correlated Yes (Kuleshov & Schrijvers, 2015) Normal-Form Correlated No Table 1: A comparison of our method against prior works in multi-agent inverse reinforcement learning and inverse game theory. The final column, labeled "Func.…”

Section: Algorithmmentioning

confidence: 99%

“…Inverse game theory. Within the context of game-theoretic and multi-agent settings, inverse game theory (Kuleshov & Schrijvers, 2015) and the inverse equilibrium problem (Waugh et al, 2011;Bestick et al, 2013) present general formulations for inferring utilities in normal form games using linear programming. In particular, these methods assume the agents rationalize to a correlated equilibrium, and Waugh et al (2011) further adopts the MaxEnt principle to disambiguate among correlated equilibria that rationalizes observed behavior.…”

Section: Related Workmentioning

confidence: 99%

“…As game theory primarily concerns itself with predicting behavior when the agent's objectives are known, a natural question to ask is the inverse problem: how can we infer the motivations of an agent by observing its behavior? This problem setting is known as multi-agent inverse reinforcement learning (MAIRL) (Yu et al, 2019), inverse game theory (IGT) (Kuleshov & Schrijvers, 2015), or the inverse equilibrium problem (Waugh et al, 2011).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Evaluating Strategic Structures in Multi-Agent Inverse Reinforcement Learning

Fu¹,

Tacchetti²,

Pérolat³

et al. 2021

jair

View full text Add to dashboard Cite

A core question in multi-agent systems is understanding the motivations for an agent's actions based on their behavior. Inverse reinforcement learning provides a framework for extracting utility functions from observed agent behavior, casting the problem as finding domain parameters which induce such a behavior from rational decision makers. We show how to efficiently and scalably extend inverse reinforcement learning to multi-agent settings, by reducing the multi-agent problem to N single-agent problems while still satisfying rationality conditions such as strong rationality. However, we observe that rewards learned naively tend to lack insightful structure, which causes them to produce undesirable behavior when optimized in games with different players from those encountered during training. We further investigate conditions under which rewards or utility functions can be precisely identified, on problem domains such as normal-form and Markov games, as well as auctions, where we show we can learn reward functions that properly generalize to new settings.

show abstract

“…For example, Honorio and Ortiz (2015) adopt a specific generative model, and optimize the fit of key parameters to the available data. Other works also employ diverse optimization techniques to uncover the payoff matrix under a best-response constraint (Kuleshov and Schrijvers 2015;Waugh, Ziebart, and Bagnell 2011;Ling, Fang, and Kolter 2018;. Models learned with such techniques have been shown to fit well to some realworld scenarios (Garg and Jaakkola 2016;.…”

Section: Multiagent Simulation For Game Model Learningmentioning

confidence: 99%

Structure Learning for Approximate Solution of Many-Player Games

Wellman

2020

AAAI

View full text Add to dashboard Cite

Games with many players are difficult to solve or even specify without adopting structural assumptions that enable representation in compact form. Such structure is generally not given and will not hold exactly for particular games of interest. We introduce an iterative structure-learning approach to search for approximate solutions of many-player games, assuming only black-box simulation access to noisy payoff samples. Our first algorithm, K-Roles, exploits symmetry by learning a role assignment for players of the game through unsupervised learning (clustering) methods. Our second algorithm, G3L, seeks sparsity by greedy search over local interactions to learn a graphical game model. Both algorithms use supervised learning (regression) to fit payoff values to the learned structures, in compact representations that facilitate equilibrium calculation. We experimentally demonstrate the efficacy of both methods in reaching quality solutions and uncovering hidden structure, on both perfectly and approximately structured game instances.

show abstract

Inverse Game Theory: Learning Utilities in Succinct Games

Cited by 24 publications

References 23 publications

Inverse Open-Loop Noncooperative Differential Games and Inverse Optimal Control

Inverse Open-Loop Noncooperative Differential Games and Inverse Optimal Control

Evaluating Strategic Structures in Multi-Agent Inverse Reinforcement Learning

Structure Learning for Approximate Solution of Many-Player Games

Contact Info

Product

Resources

About