Dynamic Interaction-Aware Scene Understanding for Reinforcement Learning in Autonomous Driving

Huegle, Maria; Kalweit, Gabriel; Werling, Moritz; Boedecker, Joschka

doi:10.1109/icra40945.2020.9197086

Cited by 23 publications

(18 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The aggregated feature vector is then propagated through another fully connected network to finally infer actions for a single ego vehicle. The authors extend their work to encode whole traffic scenes including lanes and traffic signs and compare it to using a graph convolutional network for the same task in [15]. Similarly, [16] proposes to encode the vehicles being present in the scene as graph vertices.…”

Section: B Machine Learning-based Planningmentioning

confidence: 99%

Cooperative Behavior Planning for Automated Driving using Graph Neural Networks

Klimke,

Völz,

Buchholz

2022

Preprint

View full text Add to dashboard Cite

Urban intersections are prone to delays and inefficiencies due to static precedence rules and occlusions limiting the view on prioritized traffic. Existing approaches to improve traffic flow, widely known as automatic intersection management systems, are mostly based on non-learning reservation schemes or optimization algorithms. Machine learning-based techniques show promising results in planning for a single ego vehicle. This work proposes to leverage machine learning algorithms to optimize traffic flow at urban intersections by jointly planning for multiple vehicles. Learning-based behavior planning poses several challenges, demanding for a suited input and output representation as well as large amounts of groundtruth data. We address the former issue by using a flexible graph-based input representation accompanied by a graph neural network. This allows to efficiently encode the scene and inherently provide individual outputs for all involved vehicles. To learn a sensible policy, without relying on the imitation of expert demonstrations, the cooperative planning task is phrased as a reinforcement learning problem. We train and evaluate the proposed method in an open-source simulation environment for decision making in automated driving. Compared to a first-infirst-out scheme and traffic governed by static priority rules, the learned planner shows a significant gain in flow rate, while reducing the number of induced stops. In addition to synthetic simulations, the approach is also evaluated based on real-world traffic data taken from the publicly available inD dataset.

show abstract

Section: B Machine Learning-based Planningmentioning

confidence: 99%

Cooperative Behavior Planning for Automated Driving using Graph Neural Networks

Klimke,

Völz,

Buchholz

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In the context of navigation through dynamic environments, previous works handle surrounding information with pooling [14], [15], maximum [16], concatenate [17] and sum operation [18] or the LSTM model [19], which lose the global and geometric information more or less. With the rapid development of graph convolutional networks (GCNs) and its variants [5], [6], [20], researchers tend to represent dynamic driving scenarios with many vehicles as the graphstructured information [21], [22], instead of discrete vector features. GCN is efficient and effective to aggregate surrounding perceptual information on the basis of a set of neighbors defined by an adjacency matrix.…”

Section: B Graph Representation Learningmentioning

confidence: 99%

Autonomous Navigation through intersections with Graph ConvolutionalNetworks and Conditional Imitation Learning for Self-driving Cars

Mei,

Sun,

Chen

et al. 2021

Preprint

View full text Add to dashboard Cite

In autonomous driving, navigation through unsignaled intersections with many traffic participants moving around is a challenging task. To provide a solution to this problem, we propose a novel branched network G-CIL for the navigation policy learning. Specifically, we firstly represent such dynamic environments as graph-structured data and propose an effective strategy for edge definition to aggregate surrounding information for the ego-vehicle. Then graph convolutional neural networks are used as the perception module to capture global and geometric features from the environment. To generate safe and efficient navigation policy, we further incorporate it with conditional imitation learning algorithm, to learn driving behaviors directly from expert demonstrations. Our proposed network is capable of handling a varying number of surrounding vehicles and generating optimal control actions (e.g., steering angle and throttle) according to the given high-level commands (e.g., turn left towards the global goal). Evaluations on unsignaled intersections with various traffic density demonstrate that our end-to-end trainable neural network outperforms the baselines with higher success rate and shorter navigation time.

show abstract

“…Hgle et al [3] propose a deep scenes architecture, that learns complex interaction-aware scene representations. They show the deep scenes architecture using DS and GNNs.…”

Section: A Reinforcement Learningmentioning

confidence: 99%

“…Graph neural networks (GNNs) are a class of neural networks that operate directly on graph-structured data [3]. A arXiv:2006.12576v1 [cs.LG] 22 Jun 2020 wide variety of graph neural network architectures have been proposed [4,5,6,7].…”

Section: A Graph Neural Networkmentioning

confidence: 99%

Graph Neural Networks and Reinforcement Learning for Behavior Generation in Semantic Environments

Hart

Knoll

2020

2020 IEEE Intelligent Vehicles Symposium (IV)

View full text Add to dashboard Cite

Most reinforcement learning approaches used in behavior generation utilize vectorial information as input. However, this requires the network to have a pre-defined input-size -in semantic environments this means assuming the maximum number of vehicles. Additionally, this vectorial representation is not invariant to the order and number of vehicles. To mitigate the above-stated disadvantages, we propose combining graph neural networks with actor-critic reinforcement learning. As graph neural networks apply the same network to every vehicle and aggregate incoming edge information, they are invariant to the number and order of vehicles. This makes them ideal candidates to be used as networks in semantic environments -environments consisting of objects lists. Graph neural networks exhibit some other advantages that make them favorable to be used in semantic environments. The relational information is explicitly given and does not have to be inferred. Moreover, graph neural networks propagate information through the network and can gather higher-degree information. We demonstrate our approach using a highway lane-change scenario and compare the performance of graph neural networks to conventional ones. We show that graph neural networks are capable of handling scenarios with a varying number and order of vehicles during training and application.

show abstract

Dynamic Interaction-Aware Scene Understanding for Reinforcement Learning in Autonomous Driving

Cited by 23 publications

References 14 publications

Cooperative Behavior Planning for Automated Driving using Graph Neural Networks

Cooperative Behavior Planning for Automated Driving using Graph Neural Networks

Autonomous Navigation through intersections with Graph ConvolutionalNetworks and Conditional Imitation Learning for Self-driving Cars

Graph Neural Networks and Reinforcement Learning for Behavior Generation in Semantic Environments

Contact Info

Product

Resources

About