Risk-Aware High-level Decisions for Automated Driving at Occluded Intersections with Reinforcement Learning

Kamran, Danial; Lopez, Carlos Fernandez; Lauer, Martin; Stiller, Christoph

doi:10.1109/iv47402.2020.9304606

Cited by 51 publications

(29 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As proposed in [8], we can represent the whole situation at an occluded intersection using this matrix:…”

Section: Observation and State Modelmentioning

confidence: 99%

“…Reinforcement learning (RL) has gained more attention recently in order to solve complex decision making problems in robotics. Specifically for self-driving vehicles, long term optimal policies are learned using this approach for multiple scenarios such as lane changing or merging in highways [1]- [3] or yielding at unsignalized intersections [4]- [8]. Benefitting from the high representational power provided by neural networks to learn the future return of each action, learning-based policies can provide optimal behavior which is more generic and scalable compared to POMDP-based approaches like [9] and also more intelligent than rule-based approaches [10], [11].…”

Section: Introductionmentioning

confidence: 99%

“…1: Overview of the merging scenario and the proposed distributional reinforcement learning framework for automated navigation at similar occluded intersections (background image source: [12]). generate risky behavior during training by receiving big punishments for having collisions [4], [5] or being in risky situations [6], [8], there is no guarantee that the learned policy is always safe after training. Another important challenge for the learning-based policies is to apply them in environments which are more challenging than the training environment due to higher sensor noise in perception or more severe sensor occlusions.…”

Section: Introductionmentioning

confidence: 99%

“…Another important challenge for the learning-based policies is to apply them in environments which are more challenging than the training environment due to higher sensor noise in perception or more severe sensor occlusions. Although one can try to learn a policy which reduces the amount of risk to be still safe in more challenging scenarios like [8], such a policy often is too conservative when the uncertainty is high. This dilemma between more conservative but slower policies and risky but faster policies is nicely elaborated by Isele et.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Minimizing Safety Interference for Safe and Comfortable Automated Driving with Distributional Reinforcement Learning

Kamran

Engelgeh²,

Busch³

et al. 2021

2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Self Cite

View full text Add to dashboard Cite

Despite recent advances in reinforcement learning (RL), its application in safety critical domains like autonomous vehicles is still challenging. Although punishing RL agents for risky situations can help to learn safe policies, it may also lead to highly conservative behavior. In this paper, we propose a distributional RL framework in order to learn adaptive policies that can tune their level of conservativity at run-time based on the desired comfort and utility. Using a proactive safety verification approach, the proposed framework can guarantee that actions generated from RL are fail-safe according to the worst-case assumptions. Concurrently, the policy is encouraged to minimize safety interference and generate more comfortable behavior. We trained and evaluated the proposed approach and baseline policies using a high level simulator with a variety of randomized scenarios including several corner cases which rarely happen in reality but are very crucial. In light of our experiments, the behavior of policies learned using distributional RL can be adaptive at run-time and robust to the environment uncertainty. Quantitatively, the learned distributional RL agent drives in average 8 seconds faster than the normal DQN policy and requires 83% less safety interference compared to the rule-based policy with slightly increasing the average crossing time. We also study sensitivity of the learned policy in environments with higher perception noise and show that our algorithm learns policies that can still drive reliable when the perception noise is two times higher than the training configuration for automated merging and crossing at occluded intersections.

show abstract

“…As proposed in [8], we can represent the whole situation at an occluded intersection using this matrix:…”

Section: Observation and State Modelmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Minimizing Safety Interference for Safe and Comfortable Automated Driving with Distributional Reinforcement Learning

Kamran

Engelgeh²,

Busch³

et al. 2021

2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Self Cite

View full text Add to dashboard Cite

show abstract

“…They predict the future FoV of the autonomous car over the whole planning horizon and achieve a human-like driving behavior when facing occlusions. [12] proposes to learn a driving policy at unsignalized occluded intersections via reinforcement learning, which considers the future occlusions implicitly by maximizing future reward. Other approaches like [13] and [14] incorporate active exploration to eliminate future occlusion as much as possible, to actively reduce the possibility of potential hidden TPs.…”

Section: Related Workmentioning

confidence: 99%

Reasoning about Potential Hidden Traffic Participants by Tracking Occluded Areas

Wang

Burger

Stiller

2021

2021 IEEE International Intelligent Transportation Systems Conference (ITSC)

Self Cite

View full text Add to dashboard Cite

Autonomous vehicles face big challenges guaranteeing provable safety during driving. One of the major problems is the uncertainty arising from the perception of the surrounding environment, especially due to occlusions. Recent approaches to tackle these uncertainties either minimize collision risk probabilistically or assume worst-case vehicles coming out of occluded areas. The former does not provide any safety guarantees, while the latter tends to produce overly conservative driving behavior. Human drivers, however, can reason about possible traffic participants (TPs) in occlusions with the knowledge about the street, more importantly, with the continuous observation on the changes of the field of view when moving forward. In this paper, we present an approach that imitates this human-like intelligence and can reason about all the potential TPs in occlusions. By moving forward and propagating the initial knowledge with new observations, the approach can refine the possible states of the TPs in occlusions instead of always adopting a worst-case assumption. By planning w.r.t. the set-based occupancy prediction from the refined state intervals, the vehicle can drive more efficiently under occlusions while guaranteeing safety under reasonable assumptions. The proposed method is evaluated with numerical experiments showing that all the possible hidden TPs can be covered by our refined state intervals, and with that achieving significantly more driving efficiency under occlusions while being safe.

show abstract

Verhaltensentscheidung für automatisches Fahren

Spieker,

Fischer,

Stiller

2024

Handbuch Assistiertes Und Automatisiertes Fahren

View full text Add to dashboard Cite

Risk-Aware High-level Decisions for Automated Driving at Occluded Intersections with Reinforcement Learning

Cited by 51 publications

References 11 publications

Minimizing Safety Interference for Safe and Comfortable Automated Driving with Distributional Reinforcement Learning

Minimizing Safety Interference for Safe and Comfortable Automated Driving with Distributional Reinforcement Learning

Reasoning about Potential Hidden Traffic Participants by Tracking Occluded Areas

Verhaltensentscheidung für automatisches Fahren

Contact Info

Product

Resources

About