A new prospective for Learning Automata: A machine learning approach

Abstract

In the field of Learning Automata (LA), how to design faster learning algorithms has always been a key issue. Among solutions reported in the literature, the stochastic estimator reward-inaction learning automaton (SE RI), which belongs to the Maximum Likelihood estimator based LAs, has been recognized as the fastest-optimal LA. In this paper, we first point out the limitations of the traditional Maximum Likelihood estimator (MLE) based LAs and then introduce Bayesian estimator based approach, which is demonstrated to be equivalent to Laplace smoothing of the traditional method, to overcome these limitations.

A Double Competitive Strategy-Based Learning Automata Algorithm

“…Due to the superiority of the estimator algorithms, there are many novel estimators [3][4] [5] are proposed in recent years. In 2005, Hao Ge [3] proposed a deterministic estimator based LA (Discretized Generalized Confidence Pursuit Algorithm, DGCP A) of which the estimate of each action is the upper bound of a confidence interval and extended the algorithm to stochastic estimator schemes.…”

A Parameter-Free Gradient Bayesian Two-Action Learning Automaton Scheme

“…As emphasized in section I, parameter tuning is intend to balance the trade-off between speed and accuracy. And the standard procedure of parameter tuning is pioneered in [20] and become a common practive in follow-up researches [4], [5], [7]- [9], [21].…”

“…Repeat this process 20 times, averaging over these 20 resulting values and denote it as the best resolution parameter. The value of number of successive No Error experiments is set as N E = 750, as the same value in [5], [7]- [9]. For tuning the "best" γ, In our simulation settings, for the four two-action environments, the search range of γ is from 1 to 10; For the five ten-action environments except E 7 , the search range of γ is from 1 to 20, while for E 7 , the most difficult one, the range is a little wider, from 1 to 30.…”

“…The pioneer work can be traced back to 1960s by the Soviet scholar Tsetlin [2], [3]. Since then, LA has been extensively explored and it is still under investigation as well in methodological aspects [4]- [9] as in concrete applications [10]- [17]. One intriguing property that popularize the learning automata based approaches in engineering is that LA can learn the stochastic characteristics of the external environment it interacts with, and maximize the long term reward it obtains through interacting with the environment.…”

“…For deterministic estimator based learning automata, the smallest value of the resolution parameter that yielded a hundred percent accuracy in a certain number of experiments is selected. The situation is more sophisticated when concerning the stochastic estimator based schemes [5], [8], [9], because extra configurable parameters should be set to control the perturbation added. Parameter tuning is intend to balance the trade-off between speed and accuracy.…”