Bilateral Deep Reinforcement Learning Approach for Better-than-human Car Following Model

Shi, Tianyu; Ai, Yinshuang; ElSamadisy, Omar; Abdulhai, Baher

doi:10.48550/arxiv.2203.04749

Cited by 1 publication

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Typically, the reward is a linear combination of several terms including safety, efficiency, comfort, speed-following, energy consumption, and so forth, with one of the terms in the reward function being a safety reward. The safety term is often either a large penalty (negative reward) for a crash (or a very small gap) in training (28), or a large penalty whenever the follower has a low TTC with respect to the leader (3,14). In either case, for agents trained using reward alone, the satisfaction of safety constraints is not guaranteed.…”

Section: Reinforcement Learning and Markov Decision Processesmentioning

confidence: 99%

“…Importance of Using a Target Speed Instead of a Target Gap. It is common (for example Zhu et al [3], Shi et al [14], Lin et al [28]) to formulate the efficiency part of the RL carfollowing reward as following a set target gap. In our work, we instead formulate efficiency as following the dynamic maximal safe next speed.…”

Section: Rewardsmentioning

confidence: 99%

“…In Isele et al (9), Gong et al (10), and Zhou et al (11), DRL is applied to optimize safety and efficiency. Only a few research papers tried to design a safe, efficient, and comfortable car-following model using DRL (3,(12)(13)(14).…”

mentioning

confidence: 99%

“…Second, all the existing DRL-based CFMs neglect to train and test on some common but safety-critical driving scenarios where the leader suddenly decelerates to a complete stop, and which may result in a collision. Third, DRL CFMs often focus on car-following mode, ignore speed-following mode, or do not offer a seamless switch between car-following mode and speedfollowing mode when the leader is no longer present (3,14). According to Treiber and Kesting (1), a complete car-following model must be able to seamlessly deal with such different situations as driving in free traffic, following the leader in both stationary and non-stationary situations, emergency situations when full braking is required, and approaching slow traffic caused by congestion or red traffic lights.…”

mentioning

confidence: 99%

See 3 more Smart Citations

Safe, Efficient, and Comfortable Reinforcement-Learning-Based Car-Following for AVs with an Analytic Safety Guarantee and Dynamic Target Speed

ElSamadisy

Shi

Smirnov

et al. 2023

Transportation Research Record: Journal of the Transportation R

View full text Add to dashboard Cite

Over the last decade, there has been rising interest in automated driving systems and adaptive cruise control (ACC). Controllers based on reinforcement learning (RL) are particularly promising for autonomous driving, being able to optimize a combination of criteria such as efficiency, stability, and comfort. However, RL-based controllers typically offer no safety guarantees. In this paper, we propose SECRM (the Safe, Efficient, and Comfortable RL-based car-following Model) for autonomous car-following that balances traffic efficiency maximization and jerk minimization, subject to a hard analytic safety constraint on acceleration. The acceleration constraint is derived from the criterion that the follower vehicle must have sufficient headway to be able to avoid a crash if the leader vehicle brakes suddenly. We critique safety criteria based on the time-to-collision (TTC) threshold (commonly used for RL controllers), and confirm in simulator experiments that a representative previous TTC-threshold-based RL autonomous-vehicle controller may crash (in both training and testing). In contrast, we verify that our controller SECRM is safe, in training scenarios with a wide range of leader behaviors, and in both regular-driving and emergency-braking test scenarios. We find that SECRM compares favorably in efficiency, comfort, and speed-following to both classical (non-learned) car-following controllers (intelligent driver model, Shladover, Gipps) and a representative RL-based car-following controller.

show abstract

Section: Reinforcement Learning and Markov Decision Processesmentioning

confidence: 99%