Graph Neural Networks and Reinforcement Learning for Behavior Generation in Semantic Environments

Hart, Patrick; Knoll, Alois

doi:10.1109/iv47402.2020.9304738

Cited by 32 publications

(39 citation statements)

References 12 publications

(14 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The authors extend their work to encode whole traffic scenes including lanes and traffic signs and compare it to using a graph convolutional network for the same task in [15]. Similarly, [16] proposes to encode the vehicles being present in the scene as graph vertices. However, none of the described works can handle multi-agent planning.…”

Section: B Machine Learning-based Planningmentioning

confidence: 99%

Cooperative Behavior Planning for Automated Driving using Graph Neural Networks

Klimke,

Völz,

Buchholz

2022

Preprint

View full text Add to dashboard Cite

Urban intersections are prone to delays and inefficiencies due to static precedence rules and occlusions limiting the view on prioritized traffic. Existing approaches to improve traffic flow, widely known as automatic intersection management systems, are mostly based on non-learning reservation schemes or optimization algorithms. Machine learning-based techniques show promising results in planning for a single ego vehicle. This work proposes to leverage machine learning algorithms to optimize traffic flow at urban intersections by jointly planning for multiple vehicles. Learning-based behavior planning poses several challenges, demanding for a suited input and output representation as well as large amounts of groundtruth data. We address the former issue by using a flexible graph-based input representation accompanied by a graph neural network. This allows to efficiently encode the scene and inherently provide individual outputs for all involved vehicles. To learn a sensible policy, without relying on the imitation of expert demonstrations, the cooperative planning task is phrased as a reinforcement learning problem. We train and evaluate the proposed method in an open-source simulation environment for decision making in automated driving. Compared to a first-infirst-out scheme and traffic governed by static priority rules, the learned planner shows a significant gain in flow rate, while reducing the number of induced stops. In addition to synthetic simulations, the approach is also evaluated based on real-world traffic data taken from the publicly available inD dataset.

show abstract

Section: B Machine Learning-based Planningmentioning

confidence: 99%

Cooperative Behavior Planning for Automated Driving using Graph Neural Networks

Klimke,

Völz,

Buchholz

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…G2S+BERT+RL [11] is a RL based graph-to-sequence model for natural question generation, where the answer information is utilized by an effective Deep Alignment Network and a novel bidirectional GNN is proposed to process the directed passage graph. Similarly, other work [34,42,90] investigates how to use GNNs to improve the generalization ability of RL. There are also numerous studies that leverage RL to optimize representation learning on graphs.…”

Section: Combination Gnns and Rlmentioning

confidence: 99%

Reinforced Neighborhood Selection Guided Multi-Relational Graph Neural Networks

PengHao

ZhangRuitong

DouYingtong

et al. 2021

ACM Trans. Inf. Syst.

View full text Add to dashboard Cite

Graph Neural Networks (GNNs) have been widely used for the representation learning of various structured graph data, typically through message passing among nodes by aggregating their neighborhood information via different operations. While promising, most existing GNNs oversimplify the complexity and diversity of the edges in the graph and thus are inefficient to cope with ubiquitous heterogeneous graphs, which are typically in the form of multi-relational graph representations. In this article, we propose RioGNN , a novel Reinforced, recursive, and flexible neighborhood selection guided multi-relational Graph Neural Network architecture, to navigate complexity of neural network structures whilst maintaining relation-dependent representations. We first construct a multi-relational graph, according to the practical task, to reflect the heterogeneity of nodes, edges, attributes, and labels. To avoid the embedding over-assimilation among different types of nodes, we employ a label-aware neural similarity measure to ascertain the most similar neighbors based on node attributes. A reinforced relation-aware neighbor selection mechanism is developed to choose the most similar neighbors of a targeting node within a relation before aggregating all neighborhood information from different relations to obtain the eventual node embedding. Particularly, to improve the efficiency of neighbor selecting, we propose a new recursive and scalable reinforcement learning framework with estimable depth and width for different scales of multi-relational graphs. RioGNN can learn more discriminative node embedding with enhanced explainability due to the recognition of individual importance of each relation via the filtering threshold mechanism. Comprehensive experiments on real-world graph data and practical tasks demonstrate the advancements of effectiveness, efficiency, and the model explainability, as opposed to other comparative GNN models.

show abstract

“…Some of the applied works include optimizing network routing [Almasan et al, 2019], compilers , job scheduling [Mao et al, 2019] and automatic design of transistor circuit that is transferable [Wang et al, 2020a]. Additionally, the same concept has also been extended applications such as better behavior generation in self-driving applications [Hart and Knoll, 2020], learning policies that have zero-shot transfer capabilities in robotics [Wang et al, 2018] and controlling the dynamics of a graph which can be used to represent various real-life scenarios [Meirom et al, 2021].…”

Section: Related Workmentioning

confidence: 99%

A Graph Policy Network Approach for Volt-Var Control in Power Distribution Systems

Lee¹,

Sarkar²,

Wang³

2021

Preprint

View full text Add to dashboard Cite

Volt-var control (VVC) is the problem of operating power distribution systems within healthy regimes by controlling actuators in power systems. Existing works have mostly adopted the conventional routine of representing the power systems (a graph with tree topology) as vectors to train deep reinforcement learning (RL) policies. We propose a framework that combines RL with graph neural networks and study the benefits and limitations of graph-based policy in the VVC setting. Our results show that graph-based policies converge to the same rewards asymptotically however at a slower rate when compared to vector representation counterpart. We conduct further analysis on the impact of both observations and actions: on the observation end, we examine the robustness of graph-based policy on two typical data acquisition errors in power systems, namely sensor communication failure and measurement misalignment. On the action end, we show that actuators have various impacts on the system, thus using a graph representation induced by power systems topology may not be the optimal choice. In the end, we conduct a case study to demonstrate that the choice of readout function architecture and graph augmentation can further improve training performance and robustness.

show abstract

Graph Neural Networks and Reinforcement Learning for Behavior Generation in Semantic Environments

Cited by 32 publications

References 12 publications

Cooperative Behavior Planning for Automated Driving using Graph Neural Networks

Cooperative Behavior Planning for Automated Driving using Graph Neural Networks

Reinforced Neighborhood Selection Guided Multi-Relational Graph Neural Networks

A Graph Policy Network Approach for Volt-Var Control in Power Distribution Systems

Contact Info

Product

Resources

About