Improving reinforcement learning with context detection

Silva, Bruno C. da; Basso, Eduardo W.; Perotto, Filipo Studzinski; Bazzan, Ana L. C.; Engel, Paulo Martins

doi:10.1145/1160633.1160779

Cited by 15 publications

(8 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is the trade-off of the two extremes (centralised and decentralised) formally called as DCEE framework [45]. The adjacent agents information are coordinated for better traffic flows in [21,23,[27][28][29]. Only the downstream information are added in [21,28,29] to create the state for action prediction.…”

Section: Coordination Of Intersectionsmentioning

confidence: 99%

“…The adjacent agents information are coordinated for better traffic flows in [21,23,[27][28][29]. Only the downstream information are added in [21,28,29] to create the state for action prediction. All neighbouring agents' information are added in [22,23] and neighbours hidden states are added in [46].…”

Section: Coordination Of Intersectionsmentioning

confidence: 99%

“…This partial view is used in the proposed approach to decide the preferred traffic phase. The existing approaches [21,23,[27][28][29] communicate with the neighbour intersections to get their traffic condition and consider the others traffic condition when choosing the preferred action. To avoid the communication we have used the partial views of others traffic condition.…”

Section: Coordinated Adjacent Intersection Approach Without Communicamentioning

confidence: 99%

See 2 more Smart Citations

Adaptive traffic signal control system using composite reward architecture based deep reinforcement learning

Jamil

Ganguly

Nower

2020

IET intell. transp. syst

View full text Add to dashboard Cite

The increasing traffic congestion problem can be solved by an adaptive traffic signal control (ATSC) system as it utilises real-time traffic information to control traffic signals. Recently, deep reinforcement learning (DRL) has shown its potential in solving the traffic signal timing. However, one of the main challenges of DRL is to design a proper reward function and special attention needs for a multi-objective reward design. Since the feedback to the agent depends on the reward function, a proper design of reward function is needed for fast and stable learning. In this study, the authors proposed a new reward architecture called composite reward architecture (CRA) for multi-objective ATSC to optimise multiple objectives. It calculates multiple rewards in parallel for each action and applies the majority voting method to choose the desired action. Since the traffic signal of one intersection affects the adjacent intersections, a new coordination approach is proposed to get the overall smooth traffic flow. The proposed reward architecture CRA is compared with several existing reward functions used in the literature for different traffic scenarios. The new coordinated approach is compared with the non-coordinated approach. The authors demonstrated that the proposed approaches outperform the others concerning waiting time, halting the number of vehicles, and so on.

show abstract

Section: Coordination Of Intersectionsmentioning

confidence: 99%

Section: Coordination Of Intersectionsmentioning

confidence: 99%

Section: Coordinated Adjacent Intersection Approach Without Communicamentioning

confidence: 99%

See 1 more Smart Citation

Adaptive traffic signal control system using composite reward architecture based deep reinforcement learning

Jamil

Ganguly

Nower

2020

IET intell. transp. syst

View full text Add to dashboard Cite

show abstract

“…In [19] intelligent traffic lights learn via reinforcement learning the optimal signal plan, and also adapt to changes in traffic patterns by a meaning of context detection.…”

Section: Related Workmentioning

confidence: 99%

Towards reservation-based intersection coordination: an economic approach

Vasirani

Ossowskí

2008

2008 11th International IEEE Conference on Intelligent Transportation Systems

View full text Add to dashboard Cite

Understanding and controlling a complex system like traffic is not a trivial task. To this aim, many market-based methods have been applied to the design and the management of such systems, by defining the "rules of the game" and trying to enforce a desired global outcome. We model traffic as a computational economy, where drivers trade with the intelligent infrastructure in a virtual marketplace, buying time and space to cross intersections when commuting through the city. We show how such mechanism influences the drivers' behaviour, producing benefits for both the drivers (i.e. lower average travel times) and the road network (i.e. less congestions).

show abstract

“…Based on this hypothesis, we propose, formalize and show the efficiency of a method called RL-CD, or Reinforcement Learning with Context Detection, which performs well in non-stationary environments by continuously evaluating the prediction quality of each partial model. A brief discussion of how to measure prediction quality, in order to deal with non-stationary environments, can also be found in (Silva et al, 2006). This paper is organized as follows.…”

Section: Introductionmentioning

confidence: 99%

Dealing with non-stationary environments using context detection

Silva¹,

Basso²,

Bazzan³

et al. 2006

Proceedings of the 23rd International Conference on Machine Learning - ICML '06

Self Cite

112

View full text Add to dashboard Cite

In this paper we introduce RL-CD, a method for solving reinforcement learning problems in non-stationary environments. The method is based on a mechanism for creating, updating and selecting one among several partial models of the environment. The partial models are incrementally built according to the system's capability of making predictions regarding a given sequence of observations. We propose, formalize and show the efficiency of this method both in a simple non-stationary environment and in a noisy scenario. We show that RL-CD performs better than two standard reinforcement learning algorithms and that it has advantages over methods specifically designed to cope with non-stationarity. Finally, we present known limitations of the method and future works.

show abstract

Improving reinforcement learning with context detection

Cited by 15 publications

References 4 publications

Adaptive traffic signal control system using composite reward architecture based deep reinforcement learning

Adaptive traffic signal control system using composite reward architecture based deep reinforcement learning

Towards reservation-based intersection coordination: an economic approach

Dealing with non-stationary environments using context detection

Contact Info

Product

Resources

About