End-to-End Intersection Handling using Multi-Agent Deep Reinforcement Learning

Capasso, Alessandro Paolo; Maramotti, Paolo; Dell'Eva, Anthony; Broggi, Alberto

doi:10.1109/iv48863.2021.9575135

Cited by 19 publications

(9 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast to supervised learning, RL approaches evade the requirement for large datasets by instead exploring possible actions in a simulated environment and exploiting a reward signal to learn the desired behavior. In [13], a driving policy for controlling the acceleration and steering angle is trained through RL that is applied to multiple vehicles in a common simulated environment. As there is no explicit communication between the different vehicles' policies, no cooperation is shown in traffic.…”

Section: B Machine Learning-based Planningmentioning

confidence: 99%

Cooperative Behavior Planning for Automated Driving Using Graph Neural Networks

Klimke

Völz

Buchholz

2022

2022 IEEE Intelligent Vehicles Symposium (IV)

View full text Add to dashboard Cite

Urban intersections are prone to delays and inefficiencies due to static precedence rules and occlusions limiting the view on prioritized traffic. Existing approaches to improve traffic flow, widely known as automatic intersection management systems, are mostly based on non-learning reservation schemes or optimization algorithms. Machine learning-based techniques show promising results in planning for a single ego vehicle. This work proposes to leverage machine learning algorithms to optimize traffic flow at urban intersections by jointly planning for multiple vehicles. Learning-based behavior planning poses several challenges, demanding for a suited input and output representation as well as large amounts of ground-truth data. We address the former issue by using a flexible graph-based input representation accompanied by a graph neural network. This allows to efficiently encode the scene and inherently provide individual outputs for all involved vehicles. To learn a sensible policy, without relying on the imitation of expert demonstrations, the cooperative planning task is considered as a reinforcement learning problem. We train and evaluate the proposed method in an open-source simulation environment for decision making in automated driving. Compared to a first-in-first-out scheme and traffic governed by static priority rules, the learned planner shows a significant gain in flow rate, while reducing the number of induced stops. In addition to synthetic simulations, the approach is also evaluated based on real-world traffic data taken from the publicly available inD dataset.

show abstract

Section: B Machine Learning-based Planningmentioning

confidence: 99%

Cooperative Behavior Planning for Automated Driving Using Graph Neural Networks

Klimke

Völz

Buchholz

2022

2022 IEEE Intelligent Vehicles Symposium (IV)

View full text Add to dashboard Cite

show abstract

“…Machine learning experiences high research interest for the application in prediction as well as planning for a single ego vehicle in automated driving [12]. Reinforcement learning (RL) is used for single ego behavior planning, as demonstrated for urban intersections [13] or highway lane changes [14]. In the latter work, the authors propose to use a graph-based representation of the semantic environment of the ego vehicle and a fitting GNN for processing.…”

Section: Related Workmentioning

confidence: 99%

An Enhanced Graph Representation for Machine Learning Based Automatic Intersection Management

Klimke

Jasper

Völz

et al. 2022

2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC)

View full text Add to dashboard Cite

The improvement of traffic efficiency at urban intersections receives strong research interest in the field of automated intersection management. So far, mostly nonlearning algorithms like reservation or optimization-based ones were proposed to solve the underlying multi-agent planning problem. At the same time, automated driving functions for a single ego vehicle are increasingly implemented using machine learning methods. In this work, we build upon a previously presented graph-based scene representation and graph neural network to approach the problem using reinforcement learning. The scene representation is improved in key aspects by using edge features in addition to the existing node features for the vehicles. This leads to an increased representation quality that is leveraged by an updated network architecture. The paper provides an in-depth evaluation of the proposed method against baselines that are commonly used in automatic intersection management. Compared to a traditional signalized intersection and an enhanced first-in-first-out scheme, a significant reduction of traversal duration is observed at varying traffic densities. Finally, the generalization capability of the graphbased representation is evaluated by testing the policy on intersection layouts not seen during training. The model generalizes virtually without restrictions to smaller intersection layouts and within certain limits to larger ones.

show abstract

“…In contrast to supervised learning, RL approaches evade the requirement for large datasets by instead exploring possible actions in a simulated environment and exploiting a reward signal to learn the desired behavior. In [12], a driving policy for controlling the acceleration and steering angle is trained through RL that is applied to multiple vehicles in a common simulated environment. As there is no explicit communication between the different vehicles' policies, no cooperation is shown in traffic.…”

Section: B Machine Learning-based Planningmentioning

confidence: 99%

Cooperative Behavior Planning for Automated Driving using Graph Neural Networks

Klimke,

Völz,

Buchholz

2022

Preprint

View full text Add to dashboard Cite

Urban intersections are prone to delays and inefficiencies due to static precedence rules and occlusions limiting the view on prioritized traffic. Existing approaches to improve traffic flow, widely known as automatic intersection management systems, are mostly based on non-learning reservation schemes or optimization algorithms. Machine learning-based techniques show promising results in planning for a single ego vehicle. This work proposes to leverage machine learning algorithms to optimize traffic flow at urban intersections by jointly planning for multiple vehicles. Learning-based behavior planning poses several challenges, demanding for a suited input and output representation as well as large amounts of groundtruth data. We address the former issue by using a flexible graph-based input representation accompanied by a graph neural network. This allows to efficiently encode the scene and inherently provide individual outputs for all involved vehicles. To learn a sensible policy, without relying on the imitation of expert demonstrations, the cooperative planning task is phrased as a reinforcement learning problem. We train and evaluate the proposed method in an open-source simulation environment for decision making in automated driving. Compared to a first-infirst-out scheme and traffic governed by static priority rules, the learned planner shows a significant gain in flow rate, while reducing the number of induced stops. In addition to synthetic simulations, the approach is also evaluated based on real-world traffic data taken from the publicly available inD dataset.

show abstract

End-to-End Intersection Handling using Multi-Agent Deep Reinforcement Learning

Cited by 19 publications

References 19 publications

Cooperative Behavior Planning for Automated Driving Using Graph Neural Networks

Cooperative Behavior Planning for Automated Driving Using Graph Neural Networks

An Enhanced Graph Representation for Machine Learning Based Automatic Intersection Management

Cooperative Behavior Planning for Automated Driving using Graph Neural Networks

Contact Info

Product

Resources

About