Reinforcement Learning for Autonomous Driving with Latent State Inference and Spatial-Temporal Relationships

Ma, Xiaobai; Li, Jiachen; Kochenderfer, Mykel J.; Isele, David; Fujimura, Kikuo

doi:10.1109/icra48506.2021.9562006

Cited by 43 publications

(15 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…RL has seen a resurgence due to the success of embedding deep models in various components of the learning process (see the overview in Henderson et al, 2018). Given that many agent-based systems, such as autonomous vehicle control systems and collective animal movement, are formulated in space and time, it is natural to consider RL for such problems (e.g., Ma et al, 2021, Tampuu et al, 2017. However, for many such systems it is challenging to define the local costs or rewards that control agent behavior a priori, which has led to interest in inverse reinforcement learning (IRL), whereby one uses observed system behavior to learn the underlying costs or rewards (e.g., Ng & Russell, 2000).…”

Section: Reinforcement Learningmentioning

confidence: 99%

Statistical Deep Learning for Spatial and Spatio-Temporal Data

Wikle¹,

Zammit‐Mangion²

2022

Preprint

View full text Add to dashboard Cite

Deep neural network models have become ubiquitous in recent years, and have been applied to nearly all areas of science, engineering, and industry. These models are particularly useful for data that have strong dependencies in space (e.g., images) and time (e.g., sequences). Indeed, deep models have also been extensively used by the statistical community to model spatial and spatio-temporal data through, for example, the use of multi-level Bayesian hierarchical models and deep Gaussian processes. In this review, we first present an overview of traditional statistical and machine learning perspectives for modeling spatial and spatio-temporal data, and then focus on a variety of hybrid models that have recently been developed for latent process, data, and parameter specifications. These hybrid models integrate statistical modeling ideas with deep neural network models in order to take advantage of the strengths of each modeling paradigm. We conclude by giving an overview of computational technologies that have proven useful for these hybrid models, and with a brief discussion on future research directions.

show abstract

Section: Reinforcement Learningmentioning

confidence: 99%

Statistical Deep Learning for Spatial and Spatio-Temporal Data

Wikle¹,

Zammit‐Mangion²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Graph neural networks (GNN) have achieved outstanding performance on different types of tasks in various domains. Many variants of the model architecture and message passing rules have been proposed, such as GCN [21], Spectral GCN [22], Spatial GCN [22], ChebNet [23], GraphSAGE [24], [25] and GAT [26]. In recent years, researchers attempt to leverage GNN to incorporate relational inductive biases in the learning based models to solve various real-world tasks such as traffic flow forecasting and trajectory prediction [7], [14], [27]- [31].…”

Section: B Graph Neural Networkmentioning

confidence: 99%

Spectral Temporal Graph Neural Network for Trajectory Prediction

Cao¹,

Li²,

Ma³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

An effective understanding of the contextual environment and accurate motion forecasting of surrounding agents is crucial for the development of autonomous vehicles and social mobile robots. This task is challenging since the behavior of an autonomous agent is not only affected by its own intention, but also by the static environment and surrounding dynamically interacting agents. Previous works focused on utilizing the spatial and temporal information in time domain while not sufficiently taking advantage of the cues in frequency domain. To this end, we propose a Spectral Temporal Graph Neural Network (SpecTGNN), which can capture inter-agent correlations and temporal dependency simultaneously in frequency domain in addition to time domain. SpecTGNN operates on both an agent graph with dynamic state information and an environment graph with the features extracted from context images in two streams. The model integrates graph Fourier transform, spectral graph convolution and temporal gated convolution to encode history information and forecast future trajectories. Moreover, we incorporate a multi-head spatio-temporal attention mechanism to mitigate the effect of error propagation in a long time horizon. We demonstrate the performance of SpecTGNN on two public trajectory prediction benchmark datasets, which achieves stateof-the-art performance in terms of prediction accuracy.

show abstract

“…In [64], an intention inference method of vehicles based on the Support Vector Machine (SVM) has been proposed. In [65], reinforcement learning is proposed to achieve autonomous driving at intersections. In [66], transfer learning method is used to classify intersections In [67], an autonomous driving simulation experiment is carried out through conditional imitation learning.…”

Section: )The Learning-based Strategymentioning

confidence: 99%

Autonomous Driving Strategies at Intersections: Scenarios, State-of-the-Art, and Future Outlooks

Wei

Gong

et al. 2021

2021 IEEE International Intelligent Transportation Systems Conference (ITSC)

Self Cite

View full text Add to dashboard Cite

Due to the complex and dynamic character of intersection scenarios, the autonomous driving strategy at intersections has been a difficult problem and a hot point in the research of intelligent transportation systems in recent years. This paper gives a brief summary of state-of-the-art autonomous driving strategies at intersections. Firstly, we enumerate and analyze common types of intersection scenarios, corresponding simulation platforms, as well as related datasets. Secondly, by reviewing previous studies, we have summarized characteristics of existing autonomous driving strategies and classified them into several categories. Finally, we point out problems of the existing autonomous driving strategies and put forward several valuable research outlooks.

show abstract

Reinforcement Learning for Autonomous Driving with Latent State Inference and Spatial-Temporal Relationships

Cited by 43 publications

References 22 publications

Statistical Deep Learning for Spatial and Spatio-Temporal Data

Statistical Deep Learning for Spatial and Spatio-Temporal Data

Spectral Temporal Graph Neural Network for Trajectory Prediction

Autonomous Driving Strategies at Intersections: Scenarios, State-of-the-Art, and Future Outlooks

Contact Info

Product

Resources

About