End-to-End Automated Lane-Change Maneuvering Considering Driving Style Using a Deep Deterministic Policy Gradient Algorithm

Hu, Hongyu; Lu, Ziyang; Wang, Qi; Zheng, Chengyuan

doi:10.3390/s20185443

Cited by 23 publications

(9 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [ 32 ], the DDPG algorithm was adopted to optimize torque distribution control for a multiaxle electric vehicle with in-wheel motors. In [ 33 ], an end-to-end automatic lane changing method was proposed for autonomous vehicles using the DDPG algorithm. In [ 34 ], a Proportional–Integral–Derivative (PID)-Guide controller was designed to continuously learn through RL according to the feedback of environment to achieve high-precision attitude control of spacecraft.…”

Section: Related Workmentioning

confidence: 99%

Metalearning-Based Fault-Tolerant Control for Skid Steering Vehicles under Actuator Fault Conditions

Dai

Chen

Yang

2022

Sensors

View full text Add to dashboard Cite

Using reinforcement learning (RL) for torque distribution of skid steering vehicles has attracted increasing attention recently. Various RL-based torque distribution methods have been proposed to deal with this classical vehicle control problem, achieving a better performance than traditional control methods. However, most RL-based methods focus only on improving the performance of skid steering vehicles, while actuator faults that may lead to unsafe conditions or catastrophic events are frequently omitted in existing control schemes. This study proposes a meta-RL-based fault-tolerant control (FTC) method to improve the tracking performance of vehicles in the case of actuator faults. Based on meta deep deterministic policy gradient (meta-DDPG), the proposed FTC method has a representative gradient-based metalearning algorithm workflow, which includes an offline stage and an online stage. In the offline stage, an experience replay buffer with various actuator faults is constructed to provide data for training the metatraining model; then, the metatrained model is used to develop an online meta-RL update method to quickly adapt its control policy to actuator fault conditions. Simulations of four scenarios demonstrate that the proposed FTC method can achieve a high performance and adapt to actuator fault conditions stably.

show abstract

Section: Related Workmentioning

confidence: 99%

Metalearning-Based Fault-Tolerant Control for Skid Steering Vehicles under Actuator Fault Conditions

Dai

Chen

Yang

2022

Sensors

View full text Add to dashboard Cite

show abstract

“…Additionally, r o , r c and r end represent the smooth merging reward, congestion reward and terminal reward, respectively. Specifically, r o measures the smoothness of the merging behaviour [27], which can be given by…”

Section: Rewardmentioning

confidence: 99%

Towards Robust On-Ramp Merging via Augmented Multimodal Reinforcement Learning

Bagwe¹,

Li²,

Yuan³

et al. 2022

Preprint

View full text Add to dashboard Cite

Despite the success of AI-enabled onboard perception, on-ramp merging has been one of the main challenges for autonomous driving. Due to limited sensing range of onboard sensors, a merging vehicle can hardly observe main road conditions and merge properly. By leveraging the wireless communications between connected and automated vehicles (CAVs), a merging CAV has potential to proactively obtain the intentions of nearby vehicles. However, CAVs can be prone to inaccurate observations, such as the noisy basic safety messages (BSM) and poor quality surveillance images. In this paper, we present a novel approach for Robust on-ramp merge of CAVs via Augmented and Multi-modal Reinforcement Learning, named by RAMRL. Specifically, we formulate the on-ramp merging problem as a Markov decision process (MDP) by taking driving safety, comfort driving behavior, and traffic efficiency into account. To provide reliable merging maneuvers, we simultaneously leverage BSM and surveillance images for multi-modal observation, which is used to learn a policy model through proximal policy optimization (PPO). Moreover, to improve data efficiency and provide better generalization performance, we train the policy model with augmented data (e.g., noisy BSM and noisy surveillance images). Extensive experiments are conducted with Simulation of Urban MObility (SUMO) platform under two typical merging scenarios. Experimental results demonstrate the effectiveness and efficiency of our robust on-ramp merging design.

show abstract

“…RL methodologies aim to learn a policy that maximizes the cumulative rewards received by an automatic system, as it interacts with the environment [41], [42]. One variant of RL is Deep RL, which combines DL with RL [43]. Although there are some technical differences between RL and Deep RL, for the purposes of this review, which is to differentiate IL and RL, we will use the terms RL and Deep RL interchangeably.…”

Section: Introductionmentioning

confidence: 99%

A Review of End-to-End Autonomous Driving in Urban Environments

Coelho

Oliveira

2022

IEEE Access

View full text Add to dashboard Cite

Autonomous driving in urban environments requires intelligent systems that are able to deal with complex and unpredictable scenarios. Traditional modular approaches focus on dividing the driving task into standard modules, and then use rule-based methods to connect those different modules. As such, these approaches require a significant effort to design architectures that combine all system components, and are often prone to error propagation throughout the pipeline. Recently, end-to-end autonomous driving systems have formulated the autonomous driving problem as an end-to-end learning process, with the goal of developing a policy that transforms sensory data into vehicle control commands. Despite promising results, the majority of end-to-end works in autonomous driving focus on simple driving tasks, such as lane-following, which do not fully capture the intricacies of driving in urban environments. The main contribution of this paper is to provide a detailed comparison between end-to-end autonomous driving systems that tackle urban environments. This analysis comprises two stages: a) a description of the main characteristics of the successful end-to-end approaches in urban environments; b) a quantitative comparison based on two CARLA simulator benchmarks (CoRL2017 and NoCrash). Beyond providing a detailed overview of the existent approaches, we conclude this work with the most promising aspects of end-to-end autonomous driving approaches suitable for urban environments.INDEX TERMS autonomous driving, end-to-end, imitation learning, reinforcement learning, urban environments.

show abstract

End-to-End Automated Lane-Change Maneuvering Considering Driving Style Using a Deep Deterministic Policy Gradient Algorithm

Cited by 23 publications

References 22 publications

Metalearning-Based Fault-Tolerant Control for Skid Steering Vehicles under Actuator Fault Conditions

Metalearning-Based Fault-Tolerant Control for Skid Steering Vehicles under Actuator Fault Conditions

Towards Robust On-Ramp Merging via Augmented Multimodal Reinforcement Learning

A Review of End-to-End Autonomous Driving in Urban Environments

Contact Info

Product

Resources

About