Decision-Making Strategy on Highway for Autonomous Vehicles Using Deep Reinforcement Learning

Liao, Jiangdong; Li, Teng; Tang, Xiaolin; Mu, Xingyu; Huang, B. Q.; Cao, Dongpu

doi:10.1109/access.2020.3022755

Cited by 70 publications

(25 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A dueling deep Q-network approach was demonstrated by Liao et al to make a strategy of highway decision making [126]. The method was built for lane-changing decisions to make a strategy for AVS on highways where the lateral and longitudinal motions of the host and surrounding vehicles were manipulated by a hierarchical control system.…”

Section: Decision Makingmentioning

confidence: 99%

Vision-Based Autonomous Vehicle Systems Based on Deep Learning: A Systematic Literature Review

2022

View full text Add to dashboard Cite

In the past decade, autonomous vehicle systems (AVS) have advanced at an exponential rate, particularly due to improvements in artificial intelligence, which have had a significant impact on social as well as road safety and the future of transportation systems. However, the AVS is still far away from mass production because of the high cost of sensor fusion and a lack of combination of top-tier solutions to tackle uncertainty on roads. To reduce sensor dependency and to increase manufacturing along with enhancing research, deep learning-based approaches could be the best alternative for developing practical AVS. With this vision, in this systematic review paper, we broadly discussed the literature of deep learning for AVS from the past decade for real-life implementation in core fields. The systematic review on AVS implementing deep learning is categorized into several modules that cover activities including perception analysis (vehicle detection, traffic signs and light identification, pedestrian detection, lane and curve detection, road object localization, traffic scene analysis), decision making, end-to-end controlling and prediction, path and motion planning and augmented reality-based HUD, analyzing research works from 2011 to 2021 that focus on RGB camera vision. The literature is also analyzed for final representative outcomes as visualization in augmented reality-based head-up display (AR-HUD) with categories such as early warning, road markings for improved navigation and enhanced safety with overlapping on vehicles and pedestrians in extreme visual conditions to reduce collisions. The contribution of the literature review includes detailed analysis of current state-of-the-art deep learning methods that only rely on RGB camera vision rather than complex sensor fusion. It is expected to offer a pathway for the rapid development of cost-efficient and more secure practical autonomous vehicle systems.

show abstract

Section: Decision Makingmentioning

confidence: 99%

Vision-Based Autonomous Vehicle Systems Based on Deep Learning: A Systematic Literature Review

2022

View full text Add to dashboard Cite

show abstract

“…In [ 19 ], linear programming optimization was introduced for ten plugin hybrid electric vehicles (PHEVs) to minimize costs in both grid-connected and islanded modes of operations. In [ 20 ], a decision making via DRL-based approach was conducted with an outcome that the energy management could improve the control performance; however, the vehicles were in autonomous working. The integration of modern renewable energy resources and EVs in DR program via EnergyPlan was performed in [ 21 ].…”

Section: Introductionmentioning

confidence: 99%

“…As discussed above, some existing approaches optimize economic operations by building exhausting stochastic models, e.g., [ 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 ]. Others take an easier way of considering entire EVs battery resources as passive loads, e.g., [ 27 , 28 , 29 , 30 , 31 , 32 ], burning up a valuable energy compensation resource in rush hours.…”

Section: Introductionmentioning

confidence: 99%

Heuristic Greedy Scheduling of Electric Vehicles in Vehicle-to-Grid Microgrid Owned Aggregators

Abdel-Hakim

Abo‐Elyousr

2022

Sensors

View full text Add to dashboard Cite

In on-grid microgrids, electric vehicles (EVs) have to be efficiently scheduled for cost-effective electricity consumption and network operation. The stochastic nature of the involved parameters along with their large number and correlations make such scheduling a challenging task. This paper aims at identifying pertinent innovative solutions for reducing the relevant total costs of the on-grid EVs within hybrid microgrids. To optimally scale the EVs, a heuristic greedy approach is considered. Unlike most existing scheduling methodologies in the literature, the proposed greedy scheduler is model-free, training-free, and yet efficient. The proposed approach considers different factors such as the electricity price, on-grid EVs state of arrival and departure, and the total revenue to meet the load demands. The greedy-based approach behaves satisfactorily in terms of fulfilling its objective for the hybrid microgrid system, which is established of photovoltaic, wind turbine, and a local utility grid. Meanwhile, the on-grid EVs are being utilized as an energy storage exchange location. A real time hardware-in-the-loop experimentation is comprehensively conducted to maximize the earned profit. Through different uncertainty scenarios, the ability of the proposed greedy approach to obtain a global optimal solution is assessed. A data simulator was developed for the purposes of generating evaluation datasets, which captures uncertainties in the behaviors of the system’s parameters. The greedy-based strategy is considered applicable, scalable, and efficient in terms of total operating expenditures. Furthermore, as EVs penetration became more versatile, total expenses decreased significantly. Using simulated data of an effective operational duration of 500 years, the proposed approach succeeded in cutting down the energy consumption costs by about 50–85%, beating existing state-of-the-arts results. The proposed approach is proved to be tolerant to the large amounts of uncertainties that are involved in the system’s operational data.

show abstract

“…Several highway decision-making strategies [ 12 , 13 ] have been performed with deep Reinforcement Learning; in this research, the deep Q-Learning approach is incorporated with the Faster R-CNN method so that an autonomous agent can also detect and avoid obstacles along its way while traversing. Although the deep Q-Learning and Faster R-CNN algorithms have proven to be successful for autonomous driving strategy and object classification, respectively, the fusion of these two methods for autonomous maneuver combines the benefits of these two approaches in autonomous vehicle navigation.…”

Section: Introductionmentioning

confidence: 99%

Double Deep Q-Learning and Faster R-CNN-Based Autonomous Vehicle Navigation and Obstacle Avoidance in Dynamic Environment

Issa

Das

Rahman

et al. 2021

Sensors

View full text Add to dashboard Cite

Autonomous vehicle navigation in an unknown dynamic environment is crucial for both supervised- and Reinforcement Learning-based autonomous maneuvering. The cooperative fusion of these two learning approaches has the potential to be an effective mechanism to tackle indefinite environmental dynamics. Most of the state-of-the-art autonomous vehicle navigation systems are trained on a specific mapped model with familiar environmental dynamics. However, this research focuses on the cooperative fusion of supervised and Reinforcement Learning technologies for autonomous navigation of land vehicles in a dynamic and unknown environment. The Faster R-CNN, a supervised learning approach, identifies the ambient environmental obstacles for untroubled maneuver of the autonomous vehicle. Whereas, the training policies of Double Deep Q-Learning, a Reinforcement Learning approach, enable the autonomous agent to learn effective navigation decisions form the dynamic environment. The proposed model is primarily tested in a gaming environment similar to the real-world. It exhibits the overall efficiency and effectiveness in the maneuver of autonomous land vehicles.

show abstract

Decision-Making Strategy on Highway for Autonomous Vehicles Using Deep Reinforcement Learning

Cited by 70 publications

References 30 publications

Vision-Based Autonomous Vehicle Systems Based on Deep Learning: A Systematic Literature Review

Vision-Based Autonomous Vehicle Systems Based on Deep Learning: A Systematic Literature Review

Heuristic Greedy Scheduling of Electric Vehicles in Vehicle-to-Grid Microgrid Owned Aggregators

Double Deep Q-Learning and Faster R-CNN-Based Autonomous Vehicle Navigation and Obstacle Avoidance in Dynamic Environment

Contact Info

Product

Resources

About