A new Learning Automata based approach for online tracking of event patterns

Jiang, Wen; Zhao, Chenglin; Li, Shenghong; Chen, Lawson

doi:10.1016/j.neucom.2013.08.047

Cited by 16 publications

(6 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is a decision maker which acn choose the optimal action and update its strategy through interacting with the random environment [7]. As one of the most powerful tools in adaptive learning system, LA has had a myriad of applications [9][8] [2][20][21] [6]. As illustrated in Fig.1, the process of learning is based on a learning loop involving two entities: the random environment(RE) and the LA.…”

Section: Random Environmentmentioning

confidence: 99%

See 1 more Smart Citation

A Double Competitive Strategy-Based Learning Automata Algorithm

Guo

Huang

et al. 2019

Lecture Notes in Electrical Engineering

View full text Add to dashboard Cite

Learning Automata (LA) are considered as one of the most powerful tools in the field of reinforcement learning. The family of estimator algorithms is proposed to improve the convergence rate of LA and has made great achievements. However, the estimators perform poorly on estimating the reward probabilities of actions in the initial stage of the learning process of LA. In this situation, a lot of rewards would be added to the probabilities of non-optimal actions. Thus, a large number of extra iterations are needed to compensate for these wrong rewards. In order to improve the speed of convergence, we propose a new P-model absorbing learning automaton by utilizing a double competitive strategy which is designed for updating the action probability vector. In this way, the wrong rewards can be corrected instantly. Hence, the proposed Double Competitive Algorithm overcomes the drawbacks of existing estimator algorithms. A refined analysis is presented to show the ǫ−optimality of the proposed scheme. The extensive experimental results in benchmark environments demonstrate that our proposed learning automata perform more efficiently than the most classic LA SE RI and the current fastest LA DGCP A * .

show abstract

Section: Random Environmentmentioning

confidence: 99%

“…V arshavskii and V orontsova [18] introduced the stochastic variable structure versions of LA. Since then LA has been extensively researched to develop various kinds of algorithms based on deterministic LA [6] and stochastic LA [7]. A comprehensive overview of these researches has been summarized by T hathachar [16].…”

Section: Random Environmentmentioning

confidence: 99%

A Double Competitive Strategy-Based Learning Automata Algorithm

Guo

Huang

et al. 2019

Lecture Notes in Electrical Engineering

View full text Add to dashboard Cite

show abstract

“…The literature on RWJs is extremely sparse. One example RWJ was reported in [2], and it was applied in the online tracking of spatio-temporal event patterns in [15,16].…”

Section: Analysis Of Ergodic Rws With "Jumps" (Rwj)mentioning

confidence: 99%

On the analysis of a random walk-jump chain with tree-based transitions and its applications to faulty dichotomous search

Yazidi

Oommen

2018

Sequential Analysis

View full text Add to dashboard Cite

Random Walks (RWs) have been extensively studied for more than a century [1]. These walks have traditionally been on a line, and the generalizations for two and three dimensions, have been by extending the random steps to the corresponding neighboring positions in one or many of the dimensions. Among the most popular RWs on a line are the various models for birth and death processes, renewal processes and the gambler's ruin problem. All of these RWs operate "on a discretized line", and the walk is achieved by performing small steps to the current-state's neighbor states. Indeed, it is this neighbor-step motion that renders their analyses tractable. When some of the transitions are to non-neighbour states, a formal analysis is, typically, impossible because the difference equations of the steady-state probabilities are not solvable. One endeavor on such an analysis is found in [2]. The problem is far more complex when the transitions of the walk follow an underlying tree-like structure. The analysis of RWs on a tree have received little attention, even though it is an important topic since a tree is a counterpart space representation of a line whenever there is some ordering on the nodes on the line. Nevertheless, RWs on a tree entail moving to non-neighbor states in the space, which makes the analysis involved, and in many cases, impossible. In this paper, we consider the analysis of one such fascinating RW. We demonstrate that an analysis of the chain is feasible because we can invoke the phenomenon of "time reversibility". Apart from the analysis being interesting in itself from an analytical perspective, the RW on the tree that this paper models, is a type of generalization of dichotomous search with faulty feedback about the direction of the search, rendering the real-life application of the model to be pertinent. To resolve this, we advocate the concept of "backtracking" transitions in order to efficiently explore the search space. Interestingly, it is precisely these "backtracking" transitions that naturally render the chain to be "time reversible". By doing this, we are able to bridge the gap between deterministic dichotomous search and its faulty version. The paper contains the analysis of the chain, reports some fascinating limiting properties, and also includes simulations that justify the analytic steady-state results. * The second author is grateful for the partial support provided by NSERC, the Natural Sciences and Engineering Research Council of Canada.

show abstract

“…LA play the role of an appropriate optimization tool in cases such as (i) the complex and dynamic environments where a large amount of uncertainty exists, (ii) situations where there is insufficient information about the environment, and (iii) solutions to large-scale problems categorized as NP-complete problems(see [21,22]). WSNs are one of the most significant examples in which all aforementioned cases can be found.…”

Section: Learning Automatamentioning

confidence: 99%