Investigating Value of Curriculum Reinforcement Learning in Autonomous Driving Under Diverse Road and Weather Conditions

Ozturk, Anil; Gunel, Mustafa Burak; Dagdanov, Resul; Vural, Mirac Ekim; Yurdakul, Ferhat; Dal, Melih; Üre, Nazım Kemal

doi:10.1109/ivworkshops54471.2021.9669203

Cited by 7 publications

(3 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The driving policy is evaluated quantitatively and qualitatively. Ozturk et al (2021) propose the use of curriculum reinforcement learning for autonomous driving in different road and weather conditions. This study tackled the challenge of tuning Agents for optimal performance and generalization in various driving scenarios by using curriculum reinforcement learning.…”

Section: Related Workmentioning

confidence: 99%

Deep reinforcement learning navigation via decision transformer in autonomous driving

Ge,

Zhou,

et al. 2024

Front. Neurorobot.

View full text Add to dashboard Cite

In real-world scenarios, making navigation decisions for autonomous driving involves a sequential set of steps. These judgments are made based on partial observations of the environment, while the underlying model of the environment remains unknown. A prevalent method for resolving such issues is reinforcement learning, in which the agent acquires knowledge through a succession of rewards in addition to fragmentary and noisy observations. This study introduces an algorithm named deep reinforcement learning navigation via decision transformer (DRLNDT) to address the challenge of enhancing the decision-making capabilities of autonomous vehicles operating in partially observable urban environments. The DRLNDT framework is built around the Soft Actor-Critic (SAC) algorithm. DRLNDT utilizes Transformer neural networks to effectively model the temporal dependencies in observations and actions. This approach aids in mitigating judgment errors that may arise due to sensor noise or occlusion within a given state. The process of extracting latent vectors from high-quality images involves the utilization of a variational autoencoder (VAE). This technique effectively reduces the dimensionality of the state space, resulting in enhanced training efficiency. The multimodal state space consists of vector states, including velocity and position, which the vehicle's intrinsic sensors can readily obtain. Additionally, latent vectors derived from high-quality images are incorporated to facilitate the Agent's assessment of the present trajectory. Experiments demonstrate that DRLNDT may achieve a superior optimal policy without prior knowledge of the environment, detailed maps, or routing assistance, surpassing the baseline technique and other policy methods that lack historical data.

show abstract

Section: Related Workmentioning

confidence: 99%

Deep reinforcement learning navigation via decision transformer in autonomous driving

Ge,

Zhou,

et al. 2024

Front. Neurorobot.

View full text Add to dashboard Cite

show abstract

“…All works rely only on a few manually specified curriculum stages with respective task and reward specifications for RL agents, while GNN-RL is not yet employed. Several works set the initial CL stage as an empty road without any vehicles beyond the agent [5], [6], [8], [24].…”

Section: A Related Workmentioning

confidence: 99%

Graph-Based Autonomous Driving with Traffic-Rule-Enhanced Curriculum Learning

Peiss,

Wohlgemuth,

Xue

et al. 2023

2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC)

View full text Add to dashboard Cite

Training reinforcement learning (RL) agents for motion planning in heavily constrained solution spaces may require extensive exploration, leading to long training times. In automated driving, RL agents have to learn multiple skills at once, such as collision avoidance, traffic rule adherence, and goal reaching. In this work, we decompose this complicated learning task by applying curriculum learning for the first time onto an RL agent based on graph neural networks. The curriculum's sequence of sub-tasks gradually increases the difficulty of the longitudinal and lateral motion planning problem for the agent. Each of our sub-tasks contains a set of rewards, including novel rewards for temporal-logic-based traffic rules for speed, safety distance, and braking. Unlike prior work, the agent's state is extended by map and traffic rule information. Its performance is evaluated on prerecorded, realworld traffic data instead of simulations. Our numerical results show that the multi-stage curricula let the agent learn goalseeking highway driving faster than in baseline setups trained from scratch. Including traffic rule information in both the RL state and rewards stabilizes the training and improves the agent's final goal-reaching performance.

show abstract

“…Although RL works well for specific scenarios/tasks, optimizing a policy for generalized urban driving scenarios is impractical, as the required data/experience would be enormous. To effectively separate complex tasks, recent research on safety and efficiency of RL method [10] is investigated with curriculum learning. Unlike discovering sequential structure between road and weather conditions, applying curriculum to adversarial urban driving tasks is infeasible as deciding on curriculum relationship between these complex and realistic tasks is an open research topic.…”

Section: Introductionmentioning

confidence: 99%

DeFIX: Detecting and Fixing Failure Scenarios with Reinforcement Learning in Imitation Learning Based Autonomous Driving

Dagdanov,

Eksen,

Durmus

et al. 2022

Preprint

View full text Add to dashboard Cite

Safely navigating through an urban environment without violating any traffic rules is a crucial performance target for reliable autonomous driving. In this paper, we present a Reinforcement Learning (RL) based methodology to DEtect and FIX (DeFIX) failures of an Imitation Learning (IL) agent by extracting infraction spots and re-constructing mini-scenarios on these infraction areas to train an RL agent for fixing the shortcomings of the IL approach. DeFIX is a continuous learning framework, where extraction of failure scenarios and training of RL agents are executed in an infinite loop. After each new policy is trained and added to the library of policies, a policy classifier method effectively decides on which policy to activate at each step during the evaluation. It is demonstrated that even with only one RL agent trained on failure scenario of an IL agent, DeFIX method is either competitive or does outperform state-of-the-art IL and RL based autonomous urban driving benchmarks. We trained and validated our approach on the most challenging map (Town05) of CARLA simulator which involves complex, realistic, and adversarial driving scenarios. The source code is publicly available at https://github. com/data-and-decision-lab/DeFIX

show abstract

Investigating Value of Curriculum Reinforcement Learning in Autonomous Driving Under Diverse Road and Weather Conditions

Cited by 7 publications

References 7 publications

Deep reinforcement learning navigation via decision transformer in autonomous driving

Deep reinforcement learning navigation via decision transformer in autonomous driving

Graph-Based Autonomous Driving with Traffic-Rule-Enhanced Curriculum Learning

DeFIX: Detecting and Fixing Failure Scenarios with Reinforcement Learning in Imitation Learning Based Autonomous Driving

Contact Info

Product

Resources

About