Lane-Merging Using Policy-based Reinforcement Learning and Post-Optimization

Hart, Patrick; Rychly, Leonard; Knoll, Alois

doi:10.1109/itsc.2019.8917002

Cited by 15 publications

(7 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…More than 12% of papers studied RL since it has been well investigated for its transportation-related usage. In particular, researchers intended to apply RL to make real-time safe decisions for autonomous driving vehicles [93,142,18].…”

Section: Data Synthesismentioning

confidence: 99%

“…-Post-optimization: Adding an additional safety layer after the RL to exclude unsafe actions like safe lane merging in autonomous driving [93], -Uncertainty estimation: Estimating what the agent does not know in order to avoid performing certain actions, making the agent behaviour robust to unseen observations. For example, collision avoidance for pedestrian [142], -Stability certification of RL-based controllers: Providing theoretical guarantees for RL-based controllers like [160].…”

Section: Safety In Rlmentioning

confidence: 99%

See 1 more Smart Citation

How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review

Tambon,

Laberge,

et al. 2021

Preprint

View full text Add to dashboard Cite

Context: Machine Learning (ML) has been at the heart of many innovations over the past years. However, including it in so-called "safety-critical" systems such as automotive or aeronautic has proven to be very challenging, since the shift in paradigm that ML brings completely changes traditional certification approaches.Objective: This paper aims to elucidate challenges related to the certification of ML-based safety-critical systems, as well as the solutions that are proposed in the literature to tackle them, answering the question "How to Certify Machine Learning Based Safety-critical Systems?".Method: We conduct a Systematic Literature Review (SLR) of research papers published between 2015 to 2020, covering topics related to the certification of ML systems. In total, we identified 217 papers covering topics considered to be the main pillars of ML certification: Robustness, Uncertainty, Explainability, Verification, Safe Reinforcement Learning, and Direct Certification. We analyzed the main trends and problems of each sub-field and provided summaries of the papers extracted.Results: The SLR results highlighted the enthusiasm of the community for this subject, as well as the lack of diversity in term of datasets and type of ML models. It also emphasized the need to further develop connections between academia and industries to deepen the domain study. Finally, it also illustrated This work is supported by the DEEL project CRDPJ 537462-18 funded by the National Science and Engineering Research Council of Canada (NSERC) and the Consortium for Research and Innovation in Aerospace in Québec (CRIAQ), together with its industrial partners Thales Canada inc, Bell Textron Canada Limited, CAE inc and Bombardier inc.

show abstract

Section: Data Synthesismentioning

confidence: 99%

Section: Safety In Rlmentioning

confidence: 99%

How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review

Tambon,

Laberge,

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…mixed-integer programming [22], have proven to be computationally fast for motion and strategic planning, and can act as a dual approach compared to learning-based approaches. A combination of classical and learning-based methods [23] is computationally fast and achieves safe and comfortable motions.…”

Section: A Behavior Modelingmentioning

confidence: 99%

BARK: Open Behavior Benchmarking in Multi-Agent Environments

Bernhard

Esterle

Hart

et al. 2020

2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Self Cite

View full text Add to dashboard Cite

Predicting and planning interactive behaviors in complex traffic situations presents a challenging task. Especially in scenarios involving multiple traffic participants that interact densely, autonomous vehicles still struggle to interpret situations and to eventually achieve their own driving goal. As driving tests are costly and challenging scenarios are hard to find and reproduce, simulation is widely used to develop, test, and benchmark behavior models. However, most simulations rely on datasets and simplistic behavior models for traffic participants and do not cover the full variety of real-world, interactive human behaviors. In this work, we introduce BARK, an open-source behavior benchmarking environment designed to mitigate the shortcomings stated above. In BARK, behavior models are (re-)used for planning, prediction, and simulation. A range of models is currently available, such as Monte-Carlo Tree Search and Reinforcement Learning-based behavior models. We use a public dataset and sampling-based scenario generation to show the inter-exchangeability of behavior models in BARK. We evaluate how well the models used cope with interactions and how robust they are towards exchanging behavior models. Our evaluation shows that BARK provides a suitable framework for a systematic development of behavior models.

show abstract

“…Both of these approaches achieve state-of-the-art results in continuous control tasks. Hart et al [10] applied the SAC method in a lane-merging scenario. They additionally use a post-optimization to lower the remaining collisions and to improve the ride comfort.…”

Section: Related Workmentioning

confidence: 99%

“…This is of course if one leaves aside the concerns and restrictions that come with neural networks in safetycritical applications. Using table-based approaches instead of deep neural networks or post-processing of the RL solution [10] could mitigate these restrictions.…”

Section: B Counterfactual Reasoningmentioning

confidence: 99%

Counterfactual Policy Evaluation for Decision-Making in Autonomous Driving

Hart,

Knoll

2020

Preprint

Self Cite

View full text Add to dashboard Cite

In decision-making for autonomous vehicles, we need to predict other vehicle's behaviors or learn their behavior implicitly using machine learning. However, often the predictions and learned models have errors or might be wrong altogether which can lead to dangerous situations. Therefore, decision-making algorithms should consider counterfactual reasoning such as: what would happen if the other agents will behave in a certain way? The approach we present in this paper is two-fold: First, during training, we randomly select behavior models from a behavior model pool and assign them to the other vehicles in the scenario, such as more passive or aggressive behavior models. Second, during the application, we derive several virtual worlds from the actual world that have the same initial state. In each of these worlds, we also assign behavior models from the behavior model pool to others. We then evolve these virtual worlds for a defined time-horizon. This enables us to apply counterfactual reasoning by asking what would happen if the actual world evolves as in the virtual world. In uncertain environments, this makes it possible to generate more probable risk estimates and, thus, to enable safer decision-making. We conduct studies using a lane-change scenario that shows the advantages of counterfactual reasoning using learned policies and virtual worlds to estimate their risk and performance.

show abstract

Lane-Merging Using Policy-based Reinforcement Learning and Post-Optimization

Cited by 15 publications

References 8 publications

How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review

How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review

BARK: Open Behavior Benchmarking in Multi-Agent Environments

Counterfactual Policy Evaluation for Decision-Making in Autonomous Driving

Contact Info

Product

Resources

About