Payoff-based Inhomogeneous Partially Irrational Play for potential game theoretic cooperative control: Convergence analysis

Abstract

This paper handles a kind of strategic game called potential games and develops a novel learning algorithm Payoff-based Inhomogeneous Partially Irrational Play (PIPIP). The present algorithm is based on Distributed Inhomogeneous Synchronous Learning (DISL) presented in an existing work but, unlike DISL, PIPIP allows agents to make irrational decisions with a specified probability, i.e. agents can choose an action with a low utility from the past actions stored in the memory. Due to the irrational decisions, we can prove convergence in probability of collective actions to potential function maximizers. Finally, we demonstrate the effectiveness of the present algorithm through experiments on a sensor coverage problem. It is revealed through the demonstration that the present learning algorithm successfully leads agents to around potential function maximizers even in the presence of undesirable Nash equilibria.We also see through the experiment with a moving density function that PIPIP has adaptability to environmental changes. Index Termspotential game, learning algorithm, cooperative control, multi-agent system Tatsuhiko Goto is with Toshiba Corporation, Takeshi Hatanaka(corresponding author) and Masayuki Fujita are with

NoStop: An Open Source Framework for Design and Test of Coordination Protocol for Asymmetric Threats Protection in Marine Environment

“…Another one verified in simulation is the one proposed in [12] where pareto equilibria are reached by the guards. Moreover, the protocols prosed in [6] and in [7] have been verified to converge to Nash equilibria. All those protocols have been designed for static scenarios, where the intruder is not supposed to move, i.e., it is a fixed threat.…”

A game theoretic approach for antagonistic-task coordination of underwater autonomous robots in asymmetric threats scenarios

“…We restrict the analysis to two distributed learning algorithms: the Distributed Inhomogeneous Synchronous Learning (DISL) algorithm, [5], and the Payoff-based Inhomogeneous Partially Irrational Play (PIPIP) algorithm, [6].…”

“…In [5] it is shown that any team of robots playing a constrained potential game Γ satisfying Assumption 1, and following DISL rules converges to a CNE. In [6] is shown that any team of robots playing the same game Γ, and following PIPIP rules converges to a potential maximizer which is an efficient CNE, i.e., it is the global maximizer of the potential function.…”

“…fixed area of interest in the scenario) the coverage problem has been largely studied, see e.g. [4]- [6]. Such algorithms are proven to converge to a static configuration maximizing the number of interested area covered by the robot sensors' footprint.…”

Stochastic learning in multi-agent optimization: Communication and payoff-based approaches

“…After that one needs to develop a learning procedure, which leads agents to one of these states. Among learning algorithms that are applicable to potential games and have been presented in the literature so far the following ones demonstrate an efficient performance by approaching the system to its optimum as time runs: the log-linear learning [21], [43], its payoff-based and synchronous versions [21], payoff-based inhomogeneous partially irrational play [12], [50], and adaptive Q-learning and ε-greedy decision rule [9]. All these algorithms can be applied only to discrete optimization problems and, thus, assume players to have discrete actions.…”