Accelerating autonomous learning by using heuristic selection of actions

Bianchi, Reinaldo A. C.; Ribeiro, Carlos H. C.; Costa, Anna Helena Reali

doi:10.1007/s10732-007-9031-5

Cited by 61 publications

(40 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…For example, in the Q-learning algorithm, the approximated state-action value function Q(s, a) will converge to the optimal state-value function Q * (s, a) for discrete MDPs provided that each state-action pair is visited infinitely often and the learning rate has the property of being square summable but not summable; these conditions are still applicable and not invalidated under the proposed model. Although they follow a different heuristic based approach, Bianchi et al (2008) also employ an analogous model that transparently guides the exploration behavior of an underlying reinforcement learning algorithm for increasing the rate of convergence; we refer interested reader to their work for similar theoretical results under other settings.…”

Section: Update-tree(t H)mentioning

confidence: 99%

Improving reinforcement learning by using sequence trees

2010

View full text Add to dashboard Cite

This paper proposes a novel approach to discover options in the form of stochastic conditionally terminating sequences; it shows how such sequences can be integrated into the reinforcement learning framework to improve the learning performance. The method utilizes stored histories of possible optimal policies and constructs a specialized tree structure during the learning process. The constructed tree facilitates the process of identifying frequently used action sequences together with states that are visited during the execution of such sequences. The tree is constantly updated and used to implicitly run corresponding options. The effectiveness of the method is demonstrated empirically by conducting extensive experiments on various domains with different properties.

show abstract

Section: Update-tree(t H)mentioning

confidence: 99%

Improving reinforcement learning by using sequence trees

2010

View full text Add to dashboard Cite

show abstract

“…A Heuristically Accelerated Reinforcement Learning (HARL) algorithm [3] is a way to solve a MDP problem with explicit use of a heuristic function H : S × A → for influencing the choice of actions by the learning agent. H(s, a) defines the heuristic that indicates the importance of performing action a when visiting state s. The heuristic function is strongly associated with the policy indicating which action must be taken regardless of the action-value of the other actions that could be used in the state.…”

Section: Heuristically Accelerated Reinforcement Learningmentioning

confidence: 99%

“…The first HARL algorithm proposed was the Heuristically Accelerated Q-learning (HAQL) [3], as an extension of the Q-learning algorithm [2]. The only difference between the two algorithms is that in the HAQL makes use of an heuristic function H(s, a) in the − greedy action choice rule, that can be written as:…”

Section: Heuristically Accelerated Reinforcement Learningmentioning

confidence: 99%

See 1 more Smart Citation

Using Transfer Learning to Speed-Up Reinforcement Learning: A Cased-Based Approach

Celiberto

Matsuura

Mántaras

et al. 2010

2010 Latin American Robotics Symposium and Intelligent Robotics Meeting

View full text Add to dashboard Cite

Abstract-Reinforcement Learning (RL) is a well known technique for the solution of problems where agents need to act with success in an unknown environment, learning through trial and error. However, this technique is not efficient enough to be used in applications with real world demands due to the time that the agent needs to learn. This paper investigates the use of Transfer Learning (TL) between agents to speed up the well known Q-learning Reinforcement Learning algorithm. The new approach presented here allows the use of cases in a case base as heuristics to speed up the Q-learning algorithm, combining Case-Based Reasoning (CBR) and Heuristically Accelerated Reinforcement Learning (HARL) techniques.A set of empirical evaluations were conducted in the Mountain Car Problem Domain, where the actions learned during the solution of the 2D version of the problem can be used to speed up the learning of the policies for its 3D version.The experiments were made comparing the Q-learning Reinforcement Learning algorithm, the HAQL Heuristic Accelerated Reinforcement Learning (HARL) algorithm and the TL-HAQL algorithm, proposed here. The results show that the use of a case-base for transfer learning can lead to a significant improvement in the performance of the agent, making it learn faster than using either RL or HARL methods alone.

show abstract

“…Therefore, it is necessary to study the self-learning model. At present, there are many associated researches, but most of them focus on how students should learn English by themselves, master the appropriate learning skills and efficiently use some modern technology and equipment to help them doing self-study [8][9][10][11].…”

Section: Introductionmentioning

confidence: 99%

Design and Application of an Online English Self-learning Platform

Liu

2017

Int. J. Emerg. Technol. Learn.

View full text Add to dashboard Cite

Abstract-This paper discusses the English learning methods and identifies the problems in English teaching and self-learning in China, and then based on the VB platform, it proposes building a platform to facilitate communication between teachers, learners and other participants. At the same time, it adopts the object-oriented development approach to analyze the construction method for the online English self-learning platform and uses UML language to design this platform. At last, this paper uses general testing methods to test the interfaces, functions, performances and security of the VB-based online English selflearning platform and analyzes the performances and deficiencies of the platform based on test results. Keywords-online English; self-learning; B/S; system development IntroductionInterest plays an essential role in English self-learning. Under the guidance of interest, students will take initiatives to search and learn English knowledge. Stimulated by strong interest, a student's nervous system will be very active, and their attention, active thinking and memory will be significantly improved, and as a result, they can receive information well and learn things better [1][2][3]. In order to achieve this purpose, this paper designs an online English self-learning platform, and discusses its application in detail.English learning requires students to take the initiative to learn and have a strong interest in this language. The desire for knowledge can significantly improve students' learning efficiency. If students are aware of this and intentionally carry out such self-learning activities, they will achieve better results [4][5][6]. Under the selflearning model, students will have a more positive understanding of learning and develop a strong passion for knowledge, and in the process, their learning abilities and skills will also improve [7]. Whenever encountering any problem, students will tend to think by themselves rather than learn passively under the teacher's supervision under the traditional model. According to practical experience, this model can significantly increase students' interest and prevent them from getting tired of learning under the traditional learning model. Therefore, it is necessary to study the self-learning model. At present, there are many associated researches, but most of them focus on 4

show abstract

Accelerating autonomous learning by using heuristic selection of actions

Cited by 61 publications

References 18 publications

Improving reinforcement learning by using sequence trees

Improving reinforcement learning by using sequence trees

Using Transfer Learning to Speed-Up Reinforcement Learning: A Cased-Based Approach

Design and Application of an Online English Self-learning Platform

Contact Info

Product

Resources

About