Exploring Applications of Deep Reinforcement Learning for Real-world Autonomous Driving Systems

Talpaert, Victor; Sobh, Ibrahim; Kiran, Bangalore Ravi; Mannion, Patrick; Yogamani, Senthil; El-Sallab, Ahmad; Pérez, Patrick

doi:10.5220/0007520305640572

Cited by 42 publications

(16 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multi-agent systems (MASs) are ideally suited to model a wide range of real-world problems where autonomous actors participate in distributed decision-making. Example application domains include urban and air traffic control (Yliniemi et al , 2015; Mannion et al , 2016a), autonomous vehicles (Rădulescu et al , 2018; Talpert et al , 2019), and energy systems (Walraven & Spaan, 2016; Mannion et al , 2016b; Reymond et al , 2018). Although many such problems feature multiple conflicting objectives to optimize, most MAS research focuses on agents maximizing their return w.r.t.…”

Section: Introductionmentioning

confidence: 99%

A utility-based analysis of equilibria in multi-objective normal-form games

Rădulescu¹,

Mannion²,

Zhang

et al. 2020

The Knowledge Engineering Review

Self Cite

View full text Add to dashboard Cite

In multi-objective multi-agent systems (MOMASs), agents explicitly consider the possible trade-offs between conflicting objective functions. We argue that compromises between competing objectives in MOMAS should be analyzed on the basis of the utility that these compromises have for the users of a system, where an agent’s utility function maps their payoff vectors to scalar utility values. This utility-based approach naturally leads to two different optimization criteria for agents in a MOMAS: expected scalarized returns (ESRs) and scalarized expected returns (SERs). In this article, we explore the differences between these two criteria using the framework of multi-objective normal-form games (MONFGs). We demonstrate that the choice of optimization criterion (ESR or SER) can radically alter the set of equilibria in a MONFG when nonlinear utility functions are used.

show abstract

Section: Introductionmentioning

confidence: 99%

A utility-based analysis of equilibria in multi-objective normal-form games

Rădulescu¹,

Mannion²,

Zhang

et al. 2020

The Knowledge Engineering Review

Self Cite

View full text Add to dashboard Cite

show abstract

“…The problem of ramp merging in autonomous driving is tackled in [74], where LSTM is applied to produce an internal state containing historical driving information, and DQN is applied for Q-function approximation. The authors in [75] the review the applications and address the challenges of real-world deployment of DRL in autonomous driving.…”

Section: B Aiot Perception Layer -Smart Vehiclesmentioning

confidence: 99%

Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges

Lei

Tan

Zheng

et al. 2020

IEEE Commun. Surv. Tutorials

225

View full text Add to dashboard Cite

The Internet of Things (IoT) extends the Internet connectivity into billions of IoT devices around the world, which collect and share information to reflect the status of physical world. The Autonomous Control System (ACS), on the other hand, performs control functions on the physical systems without external intervention over an extended period of time. The integration of IoT and ACS results in a new concept -autonomous IoT (AIoT). The sensors collect information on the system status, based on which intelligent agents in IoT devices as well as Edge/Fog/Cloud servers make control decisions for the actuators to react. In order to achieve autonomy, a promising method is for the intelligent agents to leverage the techniques in the field of artificial intelligence, especially reinforcement learning (RL) and deep reinforcement learning (DRL) for decision making. In this paper, we first provide comprehensive survey of the state-of-art research, and then propose a general model for the applications of RL/DRL in AIoT. Finally, the challenges and open issues for future research are identified.

show abstract

“…On the other hand, Machine Learning based approaches including Reinforcement Learning (RL) (Sutton & Barto, 2018) and Learning from Demonstration (LfD) (Argall, Chernova, Veloso, & Browning, 2009) are capable of fast, reactive control under fewer assumptions (Shalev-Shwartz, Shammah, & Shashua, 2016;Bojarski, Yeres, Choromanska, Choromanski, Firner, Jackel, & Muller, 2017;Sharifzadeh, Chiotellis, Triebel, & Cremers, 2016;You, Lu, Filev, & Tsiotras, 2019). However the training phase of these algorithms is often data-hungry (Fayjie, Hossain, Oualid, & Lee, 2018;Talpaert., Sobh., Kiran., Mannion., Yogamani., El-Sallab., & Perez., 2019) especially for those using highly expressive and complex models like deep neural networks. RL based methods also require online interaction with the environment that entails risk Santara, Naik, Ravindran, Das, Mudigere, Avancha, & Kaul, 2018).…”

Section: Introductionmentioning

confidence: 99%

MADRaS : Multi Agent Driving Simulator

Santara¹,

Rudra

Buridi

et al. 2021

jair

View full text Add to dashboard Cite

Autonomous driving has emerged as one of the most active areas of research as it has the promise of making transportation safer and more efficient than ever before. Most real-world autonomous driving pipelines perform perception, motion planning and action in a loop. In this work we present MADRaS, an open-source multi-agent driving simulator for use in the design and evaluation of motion planning algorithms for autonomous driving. Given a start and a goal state, the task of motion planning is to solve for a sequence of position, orientation and speed values in order to navigate between the states while adhering to safety constraints. These constraints often involve the behaviors of other agents in the environment. MADRaS provides a platform for constructing a wide variety of highway and track driving scenarios where multiple driving agents can be trained for motion planning tasks using reinforcement learning and other machine learning algorithms. MADRaS is built on TORCS, an open-source car-racing simulator. TORCS offers a variety of cars with different dynamic properties and driving tracks with different geometries and surface. MADRaS inherits these functionalities from TORCS and introduces support for multi-agent training, inter-vehicular communication, noisy observations, stochastic actions, and custom traffic cars whose behaviors can be programmed to simulate challenging traffic conditions encountered in the real world. MADRaS can be used to create driving tasks whose complexities can be tuned along eight axes in well-defined steps. This makes it particularly suited for curriculum and continual learning. MADRaS is lightweight and it provides a convenient OpenAI Gym interface for independent control of each car. Apart from the primitive steering-acceleration-brake control mode of TORCS, MADRaS offers a hierarchical track-position – speed control mode that can potentially be used to achieve better generalization. MADRaS uses a UDP based client server model where the simulation engine is the server and each client is a driving agent. MADRaS uses multiprocessing to run each agent as a parallel process for efficiency and integrates well with popular reinforcement learning libraries like RLLib. We show experiments on single and multi-agent reinforcement learning with and without curriculum

show abstract

Exploring Applications of Deep Reinforcement Learning for Real-world Autonomous Driving Systems

Cited by 42 publications

References 52 publications

A utility-based analysis of equilibria in multi-objective normal-form games

A utility-based analysis of equilibria in multi-objective normal-form games

Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges

MADRaS : Multi Agent Driving Simulator

Contact Info

Product

Resources

About