SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Driving

Zhou, Ming; Luo, Jun; Villela, Julian; Yang, Yaodong; Rusu, David; Miao, Jiayu; Zhang, Weinan; Alban, Montgomery; Fadakar, Iman; Chen, Zheng; Huang, Aurora Chongxi; Wen, Ying; Hassanzadeh, Kimia; Graves, Daniel; Dong, Chen; Zhu, Zhengbang; Nguyen, Nhat M.; Elsayed, Mohamed; Shao, Kun; Ahilan, Sanjeevan; Zhang, Baokuan; Wu, Jiannan; Fu, Zhengang; Rezaee, Kasra; Yadmellat, Peyman; Rohani, Mohsen; Nieves, Nicolas Perez; Ni, Yihan; Banijamali, Seyedershad; Cowen-Rivers, Alexander I.; Tian, Zheng; Palenicek, Daniel; Bou-Ammar, Haitham; Liu, Wulong; Hao, Jianye; Wang, Jun

doi:10.48550/arxiv.2010.09776

Cited by 32 publications

(39 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The vehicle needs to reinforce learning from experience, usually at the cost of collisions, to gain autonomous knowledge and achieve higher rewards. Recent research on the DRLbased driving domain focused on the long-term accumulative reward or averaged reward as the critical performance metrics [18], [19]. Nevertheless, the literature rarely paid attention to a safety measure to evaluate the autonomous performance of DRL models.…”

Section: Related Workmentioning

confidence: 99%

Self-Awareness Safety of Deep Reinforcement Learning in Road Traffic Junction Driving

Cao¹,

Yun²

2022

Preprint

View full text Add to dashboard Cite

Autonomous driving has been at the forefront of public interest, and a pivotal debate to widespread concerns is safety in the transportation system. Deep reinforcement learning (DRL) has been applied to autonomous driving to provide solutions for obstacle avoidance. However, in a road traffic junction scenario, the vehicle typically receives partial observations from the transportation environment, while DRL needs to rely on long-term rewards to train a reliable model by maximising the cumulative rewards, which may take the risk when exploring new actions and returning either a positive reward or a penalty in the case of collisions. Although safety concerns are usually considered in the design of a reward function, they are not fully considered as the critical metric to directly evaluate the effectiveness of DRL algorithms in autonomous driving. In this study, we evaluated the safety performance of three baseline DRL models (DQN, A2C, and PPO) and proposed a self-awareness module from an attention mechanism for DRL to improve the safety evaluation for an anomalous vehicle in a complex road traffic junction environment, such as intersection and roundabout scenarios, based on four metrics: collision rate, success rate, freezing rate, and total reward. Our two experimental results in the training and testing phases revealed the baseline DRL with poor safety performance, while our proposed self-awareness attention-DQN can significantly improve the safety performance in intersection and roundabout scenarios.

show abstract

Section: Related Workmentioning

confidence: 99%

Self-Awareness Safety of Deep Reinforcement Learning in Road Traffic Junction Driving

Cao¹,

Yun²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Despite those introduced in Section 4.6, simulation continues to be a significant engineering and research challenge. We have not yet seen comparable simulation granularity as that of the environments for traffic management, (e.g., SUMO [Lopez et al, 2018], Flow ) or autonomous driving (e.g., SMARTS [Zhou et al, 2020], CARLA [Dosovitskiy et al, 2017]). The opportunity is an agent-based microscopic simulation environment for ridesharing that accounts for both ride-hailing and carpool, as well as driver and passenger behavior details, e.g., price sensitivity, cancellation behavior, driver entrance/exit behavior.…”

Section: Simulation and Sim2realmentioning

confidence: 99%

Reinforcement Learning for Ridesharing: An Extended Survey

Qin¹,

Zhu²,

Ye³

2021

Preprint

View full text Add to dashboard Cite

In this paper, we present a comprehensive, in-depth survey of the literature on reinforcement learning approaches to ridesharing problems. Papers on the topics of rideshare matching, vehicle repositioning, ride-pooling, and dynamic pricing are covered. Popular data sets and open simulation environments are also introduced. Subsequently, we discuss a number of challenges and opportunities for reinforcement learning research on this important domain.

show abstract

“…PLOP [6] and Argoverse [7] use the ego trajectory in a bird's eye view map. Simaug and SMARTS [25,52] take advantage of simulation data to train the prediction model. Others [5,8,11,24,26,49] have explored multimodal inputs, such as Lidar [9,31,36,47], to aid in trajectory prediction.…”

Section: Related Workmentioning

confidence: 99%

Stepwise Goal-Driven Networks for Trajectory Prediction

Wang,

et al. 2021

Preprint

View full text Add to dashboard Cite

We propose to predict the future trajectories of observed agents (e.g., pedestrians or vehicles) by estimating and using their goals at multiple time scales. We argue that the goal of a moving agent may change over time, and modeling goals continuously provides more accurate and detailed information for future trajectory estimation. In this paper, we present a novel recurrent network for trajectory prediction, called Stepwise Goal-Driven Network (SGNet). Unlike prior work that models only a single, long-term goal, SGNet estimates and uses goals at multiple temporal scales. In particular, the framework incorporates an encoder module that captures historical information, a stepwise goal estimator that predicts successive goals into the future, and a decoder module that predicts future trajectory. We evaluate our model on three first-person traffic datasets (HEV-I, JAAD, and PIE) as well as on two bird's eye view datasets (ETH and UCY), and show that our model outperforms the state-of-the-art methods in terms of both average and final displacement errors on all datasets. Code has been made available at: https://github.com/ ChuhuaW/SGNet.pytorch.

show abstract

SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Driving

Cited by 32 publications

References 36 publications

Self-Awareness Safety of Deep Reinforcement Learning in Road Traffic Junction Driving

Self-Awareness Safety of Deep Reinforcement Learning in Road Traffic Junction Driving

Reinforcement Learning for Ridesharing: An Extended Survey

Stepwise Goal-Driven Networks for Trajectory Prediction

Contact Info

Product

Resources

About