Dealing with non-stationary environments using context detection

Silva, Bruno C. da; Basso, Eduardo W.; Bazzan, Ana L. C.; Engel, Paulo Martins

doi:10.1145/1143844.1143872

Cited by 112 publications

(85 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Choi et al developed a method called Multiple-Model Reinforcement Learning (MMRL) [8] which assumes a fixed number of dynamics and known by the designer. This assumption is not practical and unpretentious for real world problems.…”

Section: A Reinforcement Learningmentioning

confidence: 99%

“…The agent uses the model with the highest quality, E max . When there is no model that has a higher E than a pre-specified minimum match quality threshold, i.e., E min is less than the minimum quality threshold that means a new model must be created by the agent [8].…”

Section: F Reinforcement Learning With Context Detectionmentioning

confidence: 99%

“…Limitations of the problem selection for this work are mostly dictated by the RL-CD algorithm's restrictions about the environment [8]. We conduct our experiments on a grid world problem to demonstrate that 1) the algorithm works, 2) it outperforms RL-CD.…”

Section: A Setup For the Experimentsmentioning

confidence: 99%

See 2 more Smart Citations

Hierarchical Reinforcement Learning with Context Detection (HRL-CD)

Yücesoy¹,

Tümer²

2015

IJMLC

View full text Add to dashboard Cite

Abstract-A reinforcement learning (RL) agent mostly assumes environments are stationary which is not feasible on most real world problems. Most RL approaches adapt slow changes by forgetting the previous dynamics of the environment. Reinforcement learning-context detection (RL-CD) is a technique that helps determine changes of the environment's nature which the agent with the capability to learn different dynamics of the non-stationary environment. In this study we propose an autonomous agent that learns a dynamic environment by taking advantage of hierarchical reinforcement learning (HRL) and present how the hierarchical structure can be integrated into RL-CD to speed up the convergence of a policy.Index Terms-Reinforcement learning, autonomous agent, hierarchical reinforcement learning, non-stationary environment, betweenness centrality, prioritized sweeping.

show abstract

Section: A Reinforcement Learningmentioning

confidence: 99%

Section: F Reinforcement Learning With Context Detectionmentioning

confidence: 99%

See 1 more Smart Citation

Hierarchical Reinforcement Learning with Context Detection (HRL-CD)

Yücesoy¹,

Tümer²

2015

IJMLC

View full text Add to dashboard Cite

show abstract

“…In [8] a set of techniques were tried in order to improve the learning ability of the agents in a simple scenario. Performance of reinforcement learning approaches such as Q-learning and Prioritized Sweeping in non-stationary environments are compared in [13]. Co-learning is discussed in [19] (detailed here in Section 2.3).…”

Section: Real-time Optimization Of Traffic Lightsmentioning

confidence: 99%

“…We have expected Q-learning to perform bad because it is already known that it does not have a good performance in noisy and non-stationary traffic scenarios [13]. In order to test this, we have implemented a Q-learning mechanism in the traffic lights.…”

Section: Q-learning Traffic Lightsmentioning

confidence: 99%

To Adapt or Not to Adapt – Consequences of Adapting Driver and Traffic Light Agents

Bazzan

Oliveira

Klügl

et al. 2008

Adaptive Agents and Multi-Agent Systems III. Adaptation and Multi-Agent Learning

Self Cite

View full text Add to dashboard Cite

Abstract.One way to cope with the increasing traffic demand is to integrate standard solutions with more intelligent control measures. However, the result of possible interferences between intelligent control or information provision tools and other components of the overall traffic system is not easily predictable. This paper discusses the effects of integrating co-adaptive decision-making regarding route choices (by drivers) and control measures (by traffic lights). The motivation behind this is that optimization of traffic light control is starting to be integrated with navigation support for drivers. We use microscopic, agent-based modelling and simulation, in opposition to the classical network analysis, as this work focuses on the effect of local adaptation. In a scenario that exhibits features comparable to real-world networks, we evaluate different types of adaptation by drivers and by traffic lights, based on local perceptions. In order to compare the performance, we have also used a global level optimization method based on genetic algorithms.

show abstract