Car-following method based on inverse reinforcement learning for autonomous vehicle decision-making

Gao, Hongbo; Shi, Guanya; Xie, Guo; Cheng, Bo

doi:10.1177/1729881418817162

Cited by 55 publications

(29 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Lifelong or continuous learning has been a long-standing challenge for machine learning and autonomous systems. [8][9][10] Mimicking humans and animals that continuously acquire new knowledge and transfer them to new tasks throughout their lifetime, continuous learning builds an adaptive system that is capable of learning from a continuous stream of information. However, dilemma between plasticity and catastrophic forgetting 11,12 is the main challenge due to inefficiency and poor performances when relearning from scratch for new tasks.…”

Section: Related Workmentioning

confidence: 99%

Continuous reinforcement learning to adapt multi-objective optimization online for robot motion

Zhang

McLeod

Lee

et al. 2020

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

This article introduces a continuous reinforcement learning framework to enable online adaptation of multi-objective optimization functions for guiding a mobile robot to move in changing dynamic environments. The robot with this framework can continuously learn from multiple or changing environments where it encounters different numbers of obstacles moving in unknown ways at different times. Using both planned trajectories from a real-time motion planner and already executed trajectories as feedback observations, our reinforcement learning agent enables the robot to adapt motion behaviors to environmental changes. The agent contains a Q network connected to a long short-term memory network. The proposed framework is tested in both simulations and real robot experiments over various, dynamically varied task environments. The results show the efficacy of online continuous reinforcement learning for quick adaption to different, unknown, and dynamic environments.

show abstract

Section: Related Workmentioning

confidence: 99%

Continuous reinforcement learning to adapt multi-objective optimization online for robot motion

Zhang

McLeod

Lee

et al. 2020

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

show abstract

“…Additionally, tests to measure performance quantitatively during these tasks are implemented. These applications integrate field tests and simulated tests—which are correlated—and whose assessment is conducted through quantitative performance indexes for each predefined task [ 13 , 14 , 15 ].…”

Section: Introductionmentioning

confidence: 99%

Design and Implementation of Intelligent Agent Training Systems for Virtual Vehicles

Urrea

Garrido

Kern

2021

Sensors

View full text Add to dashboard Cite

This paper presents the results of the design, simulation, and implementation of a virtual vehicle. Such a process employs the Unity videogame platform and its Machine Learning-Agents library. The virtual vehicle is implemented in Unity considering mechanisms that represent accurately the dynamics of a real automobile, such as motor torque curve, suspension system, differential, and anti-roll bar, among others. Intelligent agents are designed and implemented to drive the virtual automobile, and they are trained using imitation or reinforcement. In the former method, learning by imitation, a human expert interacts with an intelligent agent through a control interface that simulates a real vehicle; in this way, the human expert receives motion signals and has stereoscopic vision, among other capabilities. In learning by reinforcement, a reward function that stimulates the intelligent agent to exert a soft control over the virtual automobile is designed. In the training stage, the intelligent agents are introduced into a scenario that simulates a four-lane highway. In the test stage, instead, they are located in unknown roads created based on random spline curves. Finally, graphs of the telemetric variables are presented, which are obtained from the automobile dynamics when the vehicle is controlled by the intelligent agents and their human counterpart, both in the training and the test track.

show abstract

“…The error existing in the single time-step prediction will accumulate and gradually increase in the sequential decision-making process, which may cause the model to reach unseen states, making the model have even worse predictions. To avoid this problem, some researchers have begun to use reinforcement learning (RL) methods [ 11 , 14 ].…”

Section: Introductionmentioning

confidence: 99%

Modeling Car-Following Behaviors and Driving Styles with Generative Adversarial Imitation Learning

Zhou

Wang

et al. 2020

Sensors

View full text Add to dashboard Cite

Building a human-like car-following model that can accurately simulate drivers’ car-following behaviors is helpful to the development of driving assistance systems and autonomous driving. Recent studies have shown the advantages of applying reinforcement learning methods in car-following modeling. However, a problem has remained where it is difficult to manually determine the reward function. This paper proposes a novel car-following model based on generative adversarial imitation learning. The proposed model can learn the strategy from drivers’ demonstrations without specifying the reward. Gated recurrent units was incorporated in the actor-critic network to enable the model to use historical information. Drivers’ car-following data collected by a test vehicle equipped with a millimeter-wave radar and controller area network acquisition card was used. The participants were divided into two driving styles by K-means with time-headway and time-headway when braking used as input features. Adopting five-fold cross-validation for model evaluation, the results show that the proposed model can reproduce drivers’ car-following trajectories and driving styles more accurately than the intelligent driver model and the recurrent neural network-based model, with the lowest average spacing error (19.40%) and speed validation error (5.57%), as well as the lowest Kullback-Leibler divergences of the two indicators used for driving style clustering.

show abstract

Car-following method based on inverse reinforcement learning for autonomous vehicle decision-making

Cited by 55 publications

References 27 publications

Continuous reinforcement learning to adapt multi-objective optimization online for robot motion

Continuous reinforcement learning to adapt multi-objective optimization online for robot motion

Design and Implementation of Intelligent Agent Training Systems for Virtual Vehicles

Modeling Car-Following Behaviors and Driving Styles with Generative Adversarial Imitation Learning

Contact Info

Product

Resources

About