Dynamic Input for Deep Reinforcement Learning in Autonomous Driving

Huegle, Maria; Kalweit, Gabriel; Mirchevska, Branka; Werling, Moritz; Boedecker, Joschka

doi:10.1109/iros40897.2019.8968560

Cited by 46 publications

(45 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While human drivers of CVs cannot make decisions based on all of the rich data made available through connectivity, connected AVs (CAVs) can quickly process such large amounts of information and thereby exhibit superior driving performance. This has been confirmed by several researchers who argued that V2X connectivity capabilities can significantly enhance the operational performance of AVs (Duell et al, 2016;Kreidieh et al, 2018;Stern et al, 2018;Huegle et al, 2019;Dong et al, 2020b;Li et al, 2020b). These researchers demonstrated that the interactions between AVs and connected humandriven vehicles (CHDVs) present several opportunities where connectivity can enrich AV performance.…”

Section: Promoting Automation Using Connectivitymentioning

confidence: 67%

“…In reinforcement learning based autonomous driving systems, for example, the level of cooperation between vehicles in the traffic stream influences the level of benefits that automated or autonomous vehicles could earn from connectivity. Huegle et al (2019) developed a deep reinforcement learning (DRL) based autonomous driving system that communicates with connected human-driven vehicles in order to execute lane-change maneuvers. The researchers showed that a dynamic input of the information obtained via connectivity is useful in training an efficient autonomous driving model.…”

Section: Promoting Automation Using Connectivitymentioning

confidence: 99%

See 1 more Smart Citation

Vehicle Connectivity and Automation: A Sibling Relationship

Chen

et al. 2020

Front. Built Environ.

View full text Add to dashboard Cite

The evolution of scientific advances has often been characterized by the amalgamation of two or more technologies. With respect to vehicle connectivity and automation, recent literature suggests that these two emerging transportation technologies can and will jointly and profoundly shape the future of transportation. However, it is not certain how the individual and synergistic benefits to be earned from these technologies is related to their prevailing levels of development. As such, it may be considered useful to revisit the primary concepts of automation and connectivity, and to identify any current and expected future synergies between them. Doing this can help generate knowledge that could be used to justify investments related to transportation systems connectivity and automation. In this discussion paper, we attempt to address some of these issues. The paper first reviews the technological concepts of systems automation and systems connectivity, and how they prospectively, from an individual and collective perspective, impact road transportation efficiency and safety. The paper also discusses the separate and common benefits of connectivity and automation, and their possible holistic effects in terms of these benefits where they overlap. The paper suggests that at the current time, the sibling relationship seems to be lopsided: vehicle connectivity has immense potential to enhance vehicle automation. Automation, on the other hand, may not significantly promote vehicle connectivity directly, at least not in the short term but possibly in the long term. The paper argues that future trends regarding market adoption of these two technologies and their relative pace of advancement or regulation, will shape the future synergies between them.

show abstract

Section: Promoting Automation Using Connectivitymentioning

confidence: 67%

Section: Promoting Automation Using Connectivitymentioning

confidence: 99%

Vehicle Connectivity and Automation: A Sibling Relationship

Chen

et al. 2020

Front. Built Environ.

View full text Add to dashboard Cite

show abstract

“…The dimension of the state representation can exponentially grow for complex scenarios when the number of vehicles n or intersecting lanes m increase. Another challenge here is that permutation of input elements can change which may cause the network to have different reactions for the same scenario with different input permutations [28].…”

Section: Scalable Reinforcement Learningmentioning

confidence: 99%

“…In order to address this problem, we use Deep sets [29] architectures that decouple the network size of machine learning algorithms from the number of input elements. Deep sets approach has already been applied to learn a lane change policy with DQN for highway scenarios in [28]. In this paper, we propose a Deep sets architecture for automated navigation at occluded merging and crossing scenarios (Figure 2).…”

Section: Scalable Reinforcement Learningmentioning

confidence: 99%

Minimizing Safety Interference for Safe and Comfortable Automated Driving with Distributional Reinforcement Learning

Kamran

Engelgeh²,

Busch³

et al. 2021

2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Despite recent advances in reinforcement learning (RL), its application in safety critical domains like autonomous vehicles is still challenging. Although punishing RL agents for risky situations can help to learn safe policies, it may also lead to highly conservative behavior. In this paper, we propose a distributional RL framework in order to learn adaptive policies that can tune their level of conservativity at run-time based on the desired comfort and utility. Using a proactive safety verification approach, the proposed framework can guarantee that actions generated from RL are fail-safe according to the worst-case assumptions. Concurrently, the policy is encouraged to minimize safety interference and generate more comfortable behavior. We trained and evaluated the proposed approach and baseline policies using a high level simulator with a variety of randomized scenarios including several corner cases which rarely happen in reality but are very crucial. In light of our experiments, the behavior of policies learned using distributional RL can be adaptive at run-time and robust to the environment uncertainty. Quantitatively, the learned distributional RL agent drives in average 8 seconds faster than the normal DQN policy and requires 83% less safety interference compared to the rule-based policy with slightly increasing the average crossing time. We also study sensitivity of the learned policy in environments with higher perception noise and show that our algorithm learns policies that can still drive reliable when the perception noise is two times higher than the training configuration for automated merging and crossing at occluded intersections.

show abstract

“…Combined with another sibling (AI technique reinforcement learning (RL)) to yield DRL, DL has also been applied to operational control and planning tasks in transportation such as traffic signal control (Wu et al., 2019) and pavement maintenance planning (Yao et al., 2020). Besides these successful applications, DRL has been used in multiple complex CAV driving control tasks including lane‐keeping and obstacle avoidance (El Sallab et al., 2017; S. Chen, Leng, et al., 2020), lane‐changing (Dong, Chen, Li, Du, et al., 2021; Dong, Chen, Li, Ha, et al., 2020; Huegle et al., 2020), merging maneuvers (Saxena et al., 2019), crossing traffic avoidance (Wang et al., 2020), and roundabout driving (J. Chen, Yuan, et al., 2019). DRL‐based controllers have a number of advantages that are significant and consequential.…”

Section: Introductionmentioning

confidence: 99%

Graph neural network and reinforcement learning for multi‐agent cooperative control of connected autonomous vehicles

Chen

Dong

et al. 2021

Computer aided Civil Eng

147

View full text Add to dashboard Cite

A connected autonomous vehicle (CAV) network can be defined as a set of connected vehicles including CAVs that operate on a specific spatial scope that may be a road network, corridor, or segment. The spatial scope constitutes an environment where traffic information is shared and instructions are issued for controlling the CAVs movements. Within such a spatial scope, high‐level cooperation among CAVs fostered by joint planning and control of their movements can greatly enhance the safety and mobility performance of their operations. Unfortunately, the highly combinatory and volatile nature of CAV networks due to the dynamic number of agents (vehicles) and the fast‐growing joint action space associated with multi‐agent driving tasks pose difficultly in achieving cooperative control. The problem is NP‐hard and cannot be efficiently resolved using rule‐based control techniques. Also, there is a great deal of information in the literature regarding sensing technologies and control logic in CAV operations but relatively little information on the integration of information from collaborative sensing and connectivity sources. Therefore, we present a novel deep reinforcement learning‐based algorithm that combines graphic convolution neural network with deep Q‐network to form an innovative graphic convolution Q network that serves as the information fusion module and decision processor. In this study, the spatial scope we consider for the CAV network is a multi‐lane road corridor. We demonstrate the proposed control algorithm using the application context of freeway lane‐changing at the approaches to an exit ramp. For purposes of comparison, the proposed model is evaluated vis‐à‐vis traditional rule‐based and long short‐term memory‐based fusion models. The results suggest that the proposed model is capable of aggregating information received from sensing and connectivity sources and prescribing efficient operative lane‐change decisions for multiple CAVs, in a manner that enhances safety and mobility. That way, the operational intentions of individual CAVs can be fulfilled even in partially observed and highly dynamic mixed traffic streams. The paper presents experimental evidence to demonstrate that the proposed algorithm can significantly enhance CAV operations. The proposed algorithm can be deployed at roadside units or cloud platforms or other centralized control facilities.

show abstract

Dynamic Input for Deep Reinforcement Learning in Autonomous Driving

Cited by 46 publications

References 16 publications

Vehicle Connectivity and Automation: A Sibling Relationship

Vehicle Connectivity and Automation: A Sibling Relationship

Minimizing Safety Interference for Safe and Comfortable Automated Driving with Distributional Reinforcement Learning

Graph neural network and reinforcement learning for multi‐agent cooperative control of connected autonomous vehicles

Contact Info

Product

Resources

About