On Learning How Players Learn: Estimation of Learning Dynamics in the Routing Game

Abstract

The routing game models congestion in transportation networks, communication networks, and other cyber physical systems in which agents compete for shared resources. We consider an online learning model of player dynamics: at each iteration, every player chooses a route (or a probability distribution over routes, which corresponds to a flow allocation over the physical network), then the joint decision of all players determines the costs of each path, which are then revealed to the players.We pose the following estimation problem: given a sequence of player decisions and the corresponding costs, we would like to estimate the learning model parameters. We consider in particular entropic mirror descent dynamics and reduce the problem to estimating the learning rates of each player.We demonstrate this method using data collected from a routing game experiment, played by human participants on a web application that we developed. When players log in, they are assigned an origin and destination on the graph. They can choose, at each iteration, a distribution over their available routes, and each player seeks to minimize her own cost. We collect a data set using this interface, then apply the proposed method to estimate the learning model parameters. We observe in particular that after an exploration phase, the joint decision of the players remains within a small distance of the Nash equilibrium. We also use the estimated model parameters to predict the flow distribution over routes, and compare these predictions to the actual distribution. Finally, we discuss some of the qualitative implications of the experiments, and give directions for future research.

Robust Power Management via Learning and Game Design

“…To account for this, a growing corpus of literature has been devoted to studying the problem of equilibrium convergence under no-regret learning, typically focusing on special classes of games (such as convex potential games, zero-sum games, routing games, etc.). Here again, most efforts have focused on the convergence of the time-averaged sequence of play because of its connection to the players' regret (Krichene et al 2015, Balandat et al 2016, Lam et al 2016). Nevertheless, the convergence of the actual sequence of play is also crucial to investigate for several reasons: First, from a practical standpoint, since the players' payoffs are determined by the sequence of chosen actions and not some fictitious variant thereof, convergence to Nash equilibrium should also be stated in terms of the actual sequence of play.…”

Finite-Time Last-Iterate Convergence for Multi-Agent Learning in Games

“…This point was only recently expounded rigorously, by Mertikopoulos et al (2018b), who show that even though follow-the-regularizedleader (another no-regret learning algorithm) converges to a Nash equilibrium in linear zero-sum games in the sense of time averages, actual joint actions orbit Nash equilibria in perpetuity. Motivated by this consideration, a growing literature (Krichene et al, 2015;Lam et al, 2016;Palaiopanos et al, 2017;Zhou et al, 2017;Mertikopoulos et al, 2017;Zhou et al, 2018;Mertikopoulos & Zhou, 2019) has aimed at obtaining last-iterate convergence results. However, all of these last-iterate convergence results are qualitative.…”

Cooperative Automated Vehicles: A Review of Opportunities and Challenges in Socially Intelligent Vehicles Beyond Networking

“…In [21], the routing game, based on non-cooperative game theory, has been explored for multiple agents to learn about the behaviour of other agents in order to make routing decisions about the use of limited shared resources (e.g., roads).…”