Use of Stochastic Automata for Parameter Self-Optimization with Multimodal Performance Criteria

Шапиро, Ян; Narendra, Kumpati S.

doi:10.1109/tssc.1969.300228

Cited by 99 publications

(40 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Each agent updates a probability distribution (or propensities) over available actions and the probability of an action is reinforced proportionally to the reward. This class of dynamics belongs to the general class of linear reward-inaction schemes and was considered first in mathematical psychology by [36] and introduced in engineering by [44]. It has also been applied in different forms in evolutionary economics, for modeling human or economic behavior [1,4,8,14,21], sociology, for modeling social network formation [46], and computer science, for learning how to reach the payoff-dominant equilibrium [50].…”

Section: Introductionmentioning

confidence: 99%

Distributed Dynamic Reinforcement of Efficient Outcomes in Multiagent Coordination and Network Formation

Chasparis

Shamma

2011

Dyn Games Appl

View full text Add to dashboard Cite

We analyze reinforcement learning under so-called "dynamic reinforcement." In reinforcement learning, each agent repeatedly interacts with an unknown environment (i.e., other agents), receives a reward, and updates the probabilities of its next action based on its own previous actions and received rewards. Unlike standard reinforcement learning, dynamic reinforcement uses a combination of long-term rewards and recent rewards to construct myopically forward looking action selection probabilities. We analyze the long-term stability of the learning dynamics for general games with pure strategy Nash equilibria and specialize the results for coordination games and distributed network formation. In this class of problems, more than one stable equilibrium (i.e., coordination configuration) may exist. We demonstrate equilibrium selection under dynamic reinforcement. In particular, we show how a single agent is able to destabilize an equilibrium in favor of another by appropriately adjusting its dynamic reinforcement parameters. We contrast the conclusions with prior game theoretic results according to which the risk-dominant equilibrium is the only robust equilibrium when agents' decisions are subject to small randomized perturbations. The analysis throughout is based on the ODE method for stochastic approximations, where a special form of perturbation in the learning dynamics allows for analyzing its behavior at the boundary points of the state space.

show abstract

Section: Introductionmentioning

confidence: 99%

Distributed Dynamic Reinforcement of Efficient Outcomes in Multiagent Coordination and Network Formation

Chasparis

Shamma

2011

Dyn Games Appl

View full text Add to dashboard Cite

show abstract

“…The results reported in this paper can be applied directly to adaptive and learning control systems [7,[12][13][14]. In such cases the outputs of the automata would correspond to decision variables or control parameters.…”

Section: R Viswanathan and Kumpati S Narendramentioning

confidence: 99%

“…Variable-structure stochastic automata have been successfully used by many authors as models of learning systems [7,[12][13][14]. The performance of such automata in game situations is investigated in this paper.…”

Section: Introductionmentioning

confidence: 99%

“…Using only a finite number of states they obtained essentially the same results as those reported in [ 1 ] for fixed-structure models with arbitrarily large number of states. The variable-structure automata used in [4] are conditionally optimal and it is only recently that unconditionally optimal schemes were developed [7,8]. In this paper the collective behavior of such optimal automata in both competitive and cooperative game situations is analyzed.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Competitive and Cooperative Games of Variable-Structure Stochastic Automata

Narendra¹

1973

Journal of Cybernetics

View full text Add to dashboard Cite

The collective behavior of variable-structure stochastic automata in competitive and cooperative game situations is investigated. In competitive games, it is demonstrated that when the automata use optimal reinforcement schemes, the Von Neumann value is achieved for games against nature as well as for two-player zero-sum games having a saddle point. For games without a saddle point the value of the game oscillates about the Von Neumann value in mixed strategies. For two-player cooperative games, the participating automata are shown to exhibit a locally optimal collective behavior and conditions are derived for achieving global optimality. The results of extensive computer studies are described briefly by considering five typical examples which reveal many interesting aspects of the collective behavior of automata.

show abstract

“…Shapiro and Narendra [6] adopted a stochastic automata model to find an optimal solution for multi-modal performance criteria. In order to improve the optimization performance of learning automata, i.e.…”

Section: Introductionmentioning

confidence: 99%

A learning automata based algorithm for optimization of continuous complex functions

Zeng

Liu

2005

Information Sciences

View full text Add to dashboard Cite

Use of Stochastic Automata for Parameter Self-Optimization with Multimodal Performance Criteria

Cited by 99 publications

References 2 publications

Distributed Dynamic Reinforcement of Efficient Outcomes in Multiagent Coordination and Network Formation

Distributed Dynamic Reinforcement of Efficient Outcomes in Multiagent Coordination and Network Formation

Competitive and Cooperative Games of Variable-Structure Stochastic Automata

A learning automata based algorithm for optimization of continuous complex functions

Contact Info

Product

Resources

About