Vehicle Control in Highway Traffic by Using Reinforcement Learning and Microscopic Traffic Simulation

Szoke, Laszlo; Aradi, Szilárd; Bécsi, Tamás; Gáspár, Péter

doi:10.1109/sisy50555.2020.9217076

Cited by 9 publications

(5 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Szoke et al [138] proposed a policy-based RL method to learn a safe driving policy that drives on a dense highway in the shortest amount of time. The state of the environment is represented by a vector with the agent's properties, lane ID, and surrounding vehicles' properties.…”

Section: Other Assistance Systemsmentioning

confidence: 99%

“…The state of the environment is represented by a vector with the agent's properties, lane ID, and surrounding vehicles' properties. SUMO simulator was used in [138] to conduct and evaluate experiments. Experiments demonstrate the capability of the learned agent to control a vehicle in a crowded and constantly changing highway environment without collision or termination events most of the time.…”

Section: Other Assistance Systemsmentioning

confidence: 99%

See 1 more Smart Citation

Exploiting Multi-Modal Fusion for Urban Autonomous Driving Using Latent Deep Reinforcement Learning

Khalil

Mouftah

2023

IEEE Trans. Veh. Technol.

View full text Add to dashboard Cite

show abstract

Section: Other Assistance Systemsmentioning

confidence: 99%

Section: Other Assistance Systemsmentioning

confidence: 99%

Exploiting Multi-Modal Fusion for Urban Autonomous Driving Using Latent Deep Reinforcement Learning

Khalil

Mouftah

2023

IEEE Trans. Veh. Technol.

View full text Add to dashboard Cite

show abstract

“…This interface provides the possibility to implement driver models with defined driving behavior. Similar functions are available through a Python interface called Traffic Control Interface (TraCI) in SUMO [20]. The whole DLL driving model is illustrated in Figure 1, which consists of three models.…”

Section: External Driving Modelmentioning

confidence: 99%

Effects of Automated Vehicle Models at the Mixed Traffic Situation on a Motorway Scenario

Fang

Tettamanti

et al. 2022

Energies

View full text Add to dashboard Cite

There is consensus in industry and academia that Highly Automated Vehicles (HAV) and Connected Automated Vehicles (CAV) will be launched into the market in the near future due to emerging autonomous driving technology. In this paper, a mixed traffic simulation framework that integrates vehicle models with different automated driving systems in the microscopic traffic simulation was proposed. Currently, some of the more mature Automated Driving Systems (ADS) functions (e.g., Adaptive Cruise Control (ACC), Lane Keeping Assistant (LKA), etc.) are already equipped in vehicles, the very next step towards a higher automated driving is represented by Level 3 vehicles and CAV which show great promise in helping to avoid crashes, ease traffic congestion, and improve the environment. Therefore, to better predict and simulate the driving behavior of automated vehicles on the motorway scenario, a virtual test framework is proposed which includes the Highway Chauffeur (HWC) and Vehicle-to-Vehicle (V2V) communication function. These functions are implemented as an external driver model in PTV Vissim. The framework uses a detailed digital twin based on the M86 road network located in southwestern Hungary, which was constructed for autonomous driving tests. With this framework, the effect of the proposed vehicle models is evaluated with the microscopic traffic simulator PTV Vissim. A case study of the different penetration rates of HAV and CAV was performed on the M86 motorway. Preliminary results presented in this paper demonstrated that introducing HAV and CAV to the current network individually will cause negative effects on traffic performance. However, a certain ratio of mixed traffic, 60% CAV and 40% Human Driver Vehicles (HDV), could reduce this negative impact. The simulation results also show that high penetration CAV has fine driving stability and less travel delay.

show abstract

“…However, no matter the applied algorithm, the environment, and the target task, none of the mentioned papers guarantee 100% performance. Our previous works also show examples of RL agents built with simple neural networks performing in highway scenarios [16], [17], but the perfect behavior can not be assured in every situation. Further examples of autonomous driving functions solved by RL, such as car-following, lane-keeping, trajectory following, merging, or driving in dense traffic, can be found in the following collection: [18]- [23].…”

Section: Related Workmentioning

confidence: 99%

Skills to Drive: Successor Features for Autonomous Highway Pilot

Szoke

Aradi

Bécsi

et al. 2022

IEEE Trans. Intell. Transport. Syst.

Self Cite

View full text Add to dashboard Cite

Reinforcement learning applications are spreading among different domains, including autonomous vehicle control. The diverse situations that can happen during, for instance, at a highway commute are infinite, and with labeled data, the perfect coverage of all use-cases sounds ambitious. However, with the complex tasks and complicated scenarios faced during an autonomous vehicle system design, the credit assignment problem arises. How to construct appropriate objectives for the artificial intelligence to learn and the preferences between the different goals also matter of the designer's choice. This work attempts to tackle the problem by utilizing successor features and providing a possible decomposition of the reward functions, guiding the agent's actions. This method makes the training easier for the agent and enables immediate, profound performance on new combined tasks. Furthermore, with the optimal composition, the desired behavior can be fine-tuned, and as an auxiliary gain, the decomposition empowers different driving styles and makes driving preferences rapidly changeable.We introduce the adaptation of FastRL algorithm to autonomous vehicle domain, meanwhile developing a stabilizing way of using Successor Features, namely DoubleFastRL. We compare our solution for a highway driving scenario with basic agents such as Q-learning having multi-objective training.

show abstract

Vehicle Control in Highway Traffic by Using Reinforcement Learning and Microscopic Traffic Simulation

Cited by 9 publications

References 14 publications

Exploiting Multi-Modal Fusion for Urban Autonomous Driving Using Latent Deep Reinforcement Learning

Exploiting Multi-Modal Fusion for Urban Autonomous Driving Using Latent Deep Reinforcement Learning

Effects of Automated Vehicle Models at the Mixed Traffic Situation on a Motorway Scenario

Skills to Drive: Successor Features for Autonomous Highway Pilot

Contact Info

Product

Resources

About