Decentralized Indirect Methods for Learning Automata Games

Abstract

We discuss the application of indirect learning methods in zero-sum and identical payoff learning automata games. We propose a novel decentralized version of the well-known pursuit learning algorithm. Such a decentralized algorithm has significant computational advantages over its centralized counterpart. The theoretical study of such a decentralized algorithm requires the analysis to be carried out in a nonstationary environment. We use a novel bootstrapping argument to prove the convergence of the algorithm. To our knowledge, this is the first time that such analysis has been carried out for zero-sum and identical payoff games. Extensive simulation studies are reported, which demonstrate the proposed algorithm's fast and accurate convergence in a variety of game scenarios. We also introduce the framework of partial communication in the context of identical payoff games of learning automata. In such games, the automata may not communicate with each other or may communicate selectively. This comprehensive framework has the capability to model both centralized and decentralized games discussed in this paper.

Swarm intelligence to coordinate decentralized learning automata in identical payoff games

“…LA selects the optimal one through interacting with an environment that provides an appropriate response (reward or penalty), which is used to update the selection probability vector of actions, as described in Algorithm 1. In recent years, LAs have been successfully applied to systems that possess incomplete knowledge about the environment, such as game playing [7], decision analysis [13], multiconstraint assignment [6], complete L-fuzzy matrix [14], and object partitioning [15]. Furthermore, LAs have been used to solve behavior recognition problems [16], stabilize distributed queuing systems [17] and continuous function optimization [18].…”

“…Each LA is responsible for learning a variable, such as a high dimensional parameter or a task in scheduling problem, etc. In order to supply a learning model that consists of many LAs that learn at the same time, decentralized learning automata (DLA) [6] [7] are presented. In general, those LAs in DLA are VSSA.…”

Decentralized and partially decentralized reinforcement learning for designing a distributed wetland system in watersheds

“…We use the Decentralized Pursuit Learning game Algorithm (DPLA) [13]. In the DPLA, each participating automata maintains a vectorˆ ( ) where k refers to the current trial and 1 ≤ ≤ is the automaton index.…”

Fast and Epsilon-Optimal Discretized Pursuit Learning Automata

“…the adaption to the environment. LAs have found a broad range of applications reported in the literature, such as game playing [2]- [4], pattern recognition [5], classification [6], knapsack problem [7], tutorial-like system [8], [9], object partitioning [10]- [13], cellular automata [14], telephony routing [15], [16], scheduling [17], minimum-spanning circle problem [18], congestion avoidance [19], function optimization [20], [21], resource allocation and assignment problems [22]- [24], automaton controller [25], control absorption columns, flexible manufacturing plants, and other applications such as dryers, vehicles, irrigation canals, multimedia network, robots, liquid-liquid extraction columns, bioreactors, distributed fuzzy logic processors, image processing, and data compression. Various LAs, their properties and applications have been reviewed in survey papers [26], [27] and books [28]- [32].…”