Reinforcement Learning with Case-Based Heuristics for RoboCup Soccer Keepaway

Celiberto, Luiz A.; Matsuura, Jackson P.; Mántaras, Ramon López de; Bianchi, Reinaldo A. C.

doi:10.1109/sbr-lars.2012.9

Cited by 7 publications

(7 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Em seguida, são apresentados os estudos de casos da metodologia proposta, onde o objetivoé identificar a influência provocada pela alteração nos parâmetros: taxa de aprendizado e fator de desconto. O estudo de caso 01 (α = 0,125 e γ = 0,9) refere-se a uma combinação de parâmetros adotada com frequência em aplicações do AR no futebol de robôs (Celiberto Jr et al, 2012;Berton and Bianchi, 2015). Já outros estudos de casos, são variações das configurações de α e γ, a fim de ilustrar a capacidade da interface em mostrar influência experimental desses parâmetros no aprendizado.…”

Section: Metodologia Experimentalunclassified

“…Já o trabalho de Fabro et al (2013), compreende uma discretização dos estados com base no posicionamento dos jogadores em relação a bola e aos oponentes. Em outros trabalhos, heurísticas são aplicadas na busca por desempenho do algoritmo de AR (Celiberto Jr et al, 2012). No entanto, apesar da relevância do AR para o futebol de robôs, a literatura ainda carece de ferramentas para a execução de algoritmos de aprendizado e a análise de jogos da Simulação 2D.…”

Section: Introductionunclassified

See 1 more Smart Citation

Desenvolvimento de uma Interface Gráfica Didática para o Ensino de Aprendizado por Reforço com Futebol de Robôs

Jesus

Ottoni

2020

Anais Do Congresso Brasileiro De Automática 2020

View full text Add to dashboard Cite

Este artigo apresenta uma interface gr´áfica para a realização de experimentos de Aprendizado por Reforco (AR) aplicado ao domínio do futebol simulado. O ambiente de aprendizado proposto permite ao usuário trabalhar com os principais parâmetros do AR, condicionado a diferentes combinações, como: taxa de aprendizado e fator de desconto. Osresultados obtidos, a partir de uma metodologia experimental proposta, evidenciam que é possível realizar um estudo comparativo de rendimento do aprendizado no futebol de robôs a partir de dados gerados pela interface gráfica apresentada.

show abstract

Section: Metodologia Experimentalunclassified

Section: Introductionunclassified

Desenvolvimento de uma Interface Gráfica Didática para o Ensino de Aprendizado por Reforço com Futebol de Robôs

Jesus

Ottoni

2020

Anais Do Congresso Brasileiro De Automática 2020

View full text Add to dashboard Cite

show abstract

“…The approach has proven effective in a number of domains. Celiberto et al [2] proposed a heuristic exploration function through case-based reasoning and Bianchi et al [3] transferred existing similar cases or trajectories to heuristics. Bianchi et al [4] also proved the convergence of heuristic exploration and the boundary of error estimation in theoretical analysis.…”

Section: Introductionmentioning

confidence: 99%

A Study of Qualitative Knowledge-Based Exploration for Continuous Deep Reinforcement Learning

Cao

Liu

et al. 2017

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYAs an important method to solve sequential decisionmaking problems, reinforcement learning learns the policy of tasks through the interaction with environment. But it has difficulties scaling to largescale problems. One of the reasons is the exploration and exploitation dilemma which may lead to inefficient learning. We present an approach that addresses this shortcoming by introducing qualitative knowledge into reinforcement learning using cloud control systems to represent 'if-then' rules. We use it as the heuristics exploration strategy to guide the action selection in deep reinforcement learning. Empirical evaluation results show that our approach can make significant improvement in the learning process.

show abstract

“…Similar combinations of RL and CBR have already been successfully applied to various decision problems, e.g. dynamic inventory control (Jiang and Sheng, 2009), RoboCup Soccer (Celiberto et al, 2012) and control of a simulated mountain car (Bianchi et al, 2015). For example, Jiang and Sheng (2009) propose an effective case-based RL algorithm, where CBR is used for analysing the similarity between different states of a dynamic multi-agent RL problem.…”

Section: Introductionmentioning

confidence: 99%

“…For example, Jiang and Sheng (2009) propose an effective case-based RL algorithm, where CBR is used for analysing the similarity between different states of a dynamic multi-agent RL problem. Celiberto et al (2012) and Bianchi et al (2015) develop transfer learning algorithms that transfer knowledge between similar learning tasks whilst using CBR to make this process faster. However, the only example of applying this methodology in the wireless communications domain is proposed by us in (Morozs et al, 2013).…”

Section: Introductionmentioning

confidence: 99%

Cognitive spectrum management in dynamic cellular environments: A case-based Q-learning approach

Morozs

Clarke

Grace

2016

Engineering Applications of Artificial Intelligence

View full text Add to dashboard Cite

This paper examines how novel cellular system architectures and intelligent spectrum management techniques can be used to play a key role in accommodating the exponentially increasing demand for mobile data capacity in the near future. A significant challenge faced by the artificial intelligence methods applied to such flexible wireless communication systems is their dynamic nature, e.g. network topologies that change over time. This paper proposes an intelligent case-based Q-learning method for dynamic spectrum access (DSA) which improves and stabilises the performance of cognitive cellular systems with dynamic topologies. The proposed approach is the combination of classical distributed Q-learning and a novel implementation of case-based reasoning which aims to facilitate a number of learning processes running in parallel. Large scale simulations of a stadium small cell network show that the proposed case-based Q-learning approach achieves a consistent improvement in the system quality of service (QoS) under dynamic and asymmetric network topology and traffic load conditions. Simulations of a secondary spectrum sharing scenario show that the cognitive cellular system that employs the proposed case-based Q-learning DSA scheme is able to accommodate a 28-fold increase in the total primary and secondary system throughput, but with no need for additional spectrum and with no degradation in the primary user QoS.

show abstract

Reinforcement Learning with Case-Based Heuristics for RoboCup Soccer Keepaway

Cited by 7 publications

References 26 publications

Desenvolvimento de uma Interface Gráfica Didática para o Ensino de Aprendizado por Reforço com Futebol de Robôs

Desenvolvimento de uma Interface Gráfica Didática para o Ensino de Aprendizado por Reforço com Futebol de Robôs

A Study of Qualitative Knowledge-Based Exploration for Continuous Deep Reinforcement Learning

Cognitive spectrum management in dynamic cellular environments: A case-based Q-learning approach

Contact Info

Product

Resources

About