Speed harmonisation and merge control using connected automated vehicles on a highway lane closure: a reinforcement learning approach

Ko, Byungjin; Ryu, Seunghan; Park, Byungkyu Brian; Son, Sang H.

doi:10.1049/iet-its.2019.0709

Cited by 16 publications

(11 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Reinforcement learning has greatly simplified verification and validation of performance of algorithms and models for ITS [21][22][23]; however, there are challenges in the definition of a suitable training environment in accordance with real world parameters as well as the definition of meaningful reward functions for taking multiple actions in a given training, which are typical scenarios of ITSs (e.g. slow the car down while turning to the left).…”

Section: A Overview Of Intelligent Transportation Systemsmentioning

confidence: 99%

A Sensory Similarities Approach to Load Disaggregation of Charging Stations in Internet of Electric Vehicles

et al. 2021

View full text Add to dashboard Cite

Intelligent transportation systems (ITSs) have become popular in recent years as an essential requirement for safer and more efficient transportation systems. Internet of Electric vehicles (IoEV) as well as their hybrid forms provide an ideal means of supporting sustainability within an ITS. The control of charging/discharging of EV is still a challenge, despite the tremendous research progress to date in the field. In this paper, the use of charging station data and binary vectorization are proposed in order to provide timely insights on the dynamic behavior of charging processes. A Bag-of-Power-States model has been created for similarity measurement of charging stations within given time periods. The results of experimentations using synthetic data have shown that the proposed Bag-of-Power-States model is computationally feasible and provides useful results for optimizing the scheduling of power supply to charging stations that may be located across a wide range of distances, over the same period of time.

show abstract

Section: A Overview Of Intelligent Transportation Systemsmentioning

confidence: 99%

A Sensory Similarities Approach to Load Disaggregation of Charging Stations in Internet of Electric Vehicles

et al. 2021

View full text Add to dashboard Cite

show abstract

“…It owns an online network to calculate online Q-value q (s t , a t ; ) and the target network to calculate target Q-value q(s t +1 , a t +1 ; ′ ). The t +1 is defined as a t ; )]∇ q (s, a; ) (10) According to Equation 10, the maximum of the target Qvalue is always used to be selected, which leads to the overoptimisation of the target value estimate. DDQN [40] network is designed to solve the target value over-optimisation problem by choosing the action that owns max Q-value from the online Q-network and its Q-value from the target Q-network as the target Q-value.…”

Section: Algorithm Designmentioning

confidence: 99%

“…Recently, the deep reinforcement learning (DRL) method has been dramatically promoted and shows a great prospect to solve a variety of problems in the UGV area, including dynamic control [8], path planning [9], and motion harmonisation [10]. Compared to classic autonomous drive systems with environment perception, path planning, and dynamics control modules [11], the DRL method combines deep neural networks with a reinforcement learning frame.…”

Section: Introductionmentioning

confidence: 99%

A centralised training algorithm with D3QN for scalable regular unmanned ground vehicle formation maintenance

Yuan

2021

IET Intelligent Trans Sys

View full text Add to dashboard Cite

The unmanned ground vehicle (UGV) has been widely used to accomplish various missions in civilian or military environments. Formation of the UGVs group is an important technique to support the broad applications of multi-functional UGVs. This study proposes a scalable regular UGV formation maintenance (SRUFM) algorithm based on deep reinforcement learning (DRL), which aims to use a unified DRL framework to improve the lateral control and longitudinal control performance of UGV in different situations of the formation. Based on the tailored design of state, action space, related inter-vehicle information, and the dueling double deep Q-network (D3QN), SRUFM outperforms nature deep Q-network and double deep Q-network in the exploration efficiency and the convergence speed in same CARLA training environments with fixed size formation. Furthermore, when the formation's scale is extended with similar initialisation conditions, the SRUFM can still get a nearly 90% success rate to execute all experiment formation maintenance missions after 4000 episodes of training. Each UGV in the formation can keep distance within the upper and lower error threshold of 0.15 m. The simulation experiments show that the proposed centralised training frame with D3QN is suitable to solve scalable regular UGV formation maintenance missions.

show abstract

“…In Wu et al 36 and Zhao et al, 37 DRL was used to achieve autonomous driving of V2X (vehicle-to-everything) scenario. Speed harmonization and merge control on CAVs (connected automated vehicles) was achieved by DRL in Ko et al 38 Although their control objects are different, these articles have a common feature, which is to use the DRL method to determine a specific control policy of each agent. [39][40][41] It is worth noting that because the train is limited by the track, its movement dimension is definite and it is better controlled than other types of vehicles.…”

Section: Related Workmentioning

confidence: 99%

Intelligent train control for cooperative train formation: A deep reinforcement learning approach

Zhang

Zhao

Zhang

et al. 2021

Proceedings of the Institution of Mechanical Engineers, Part I:

View full text Add to dashboard Cite

Considering the intelligent train control problem in long-term evolution for metro system, a new train-to-train communication-based train control system is proposed, where the cooperative train formation technology is introduced for realizing a more flexible train operation mode. To break the limitation of centralized train control, a pre-exploration-based two-stage deep Q-learning algorithm is adopted in the cooperative train formation, which is one of the first intelligent approaches for urban railway formation control. In addition, a comfort-considered algorithm is given, where optimization measures are taken for providing superior passenger experience. The simulation results illustrate that the optimized algorithm has a smoother jerk curve during the train control process, and the passenger comfort can be improved. Furthermore, the proposed algorithm can effectively accomplish the train control task in the multi-train tracking scenarios, and meet the control requirements of the cooperative formation system.

show abstract

Speed harmonisation and merge control using connected automated vehicles on a highway lane closure: a reinforcement learning approach

Cited by 16 publications

References 31 publications

A Sensory Similarities Approach to Load Disaggregation of Charging Stations in Internet of Electric Vehicles

A Sensory Similarities Approach to Load Disaggregation of Charging Stations in Internet of Electric Vehicles

A centralised training algorithm with D3QN for scalable regular unmanned ground vehicle formation maintenance

Intelligent train control for cooperative train formation: A deep reinforcement learning approach

Contact Info

Product

Resources

About