QUOTA: The Quantile Option Architecture for Reinforcement Learning

Zhang, Shangtong; Mavrin, Borislav; Kong, Linglong; Liu, Bo; Yao, Hengshuai

doi:10.48550/arxiv.1811.02073

Cited by 3 publications

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This requires a representation of the risks in decision making with a principled way of evaluating the risks to decide which risk level for the vehicle to undertake. Experiments by Zhang et al (2018) showed that considering no risks but only acting according to the mean of return as in traditional RL is not always the best decision and can fail to discover the optimal policy. In contrast, acting according to different levels of risks can find the optimal policy in environments with a diverse range of different transition and reward dynamics.…”

Section: Temporal Questions and Question Hierarchiesmentioning

confidence: 99%

Towards Safe, Explainable, and Regulated Autonomous Driving

Atakishiyev¹,

Salameh²,

Yao³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

There has been growing interest in the development and deployment of autonomous vehicles on modern road networks over the last few years, encouraged by the empirical successes of powerful artificial intelligence approaches (AI), especially in the applications of deep and reinforcement learning. However, there have been several road accidents with "autonomous" cars that prevent this technology from being publicly acceptable at a wider level. As AI is the main driving force behind the intelligent navigation systems of such vehicles, both the stakeholders and transportation jurisdictions require their AI-driven software architecture to be safe, explainable, and regulatory compliant. We present a framework that integrates autonomous control, explainable AI architecture, and regulatory compliance to address this issue and further provide several conceptual models from this perspective, to help guide future research directions.

show abstract

Section: Temporal Questions and Question Hierarchiesmentioning

confidence: 99%

Towards Safe, Explainable, and Regulated Autonomous Driving

Atakishiyev¹,

Salameh²,

Yao³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Sample-based Distributional Policy Gradient

Singh¹,

Lee²,

Chen³

2020

Preprint

View full text Add to dashboard Cite

Damped Anderson Mixing for Deep Reinforcement Learning: Acceleration, Convergence, and Stabilization

Sun¹,

Liu²,

Zhao³

et al. 2021

Preprint

View full text Add to dashboard Cite

Anderson mixing has been heuristically applied to reinforcement learning (RL) algorithms for accelerating convergence and improving the sampling efficiency of deep RL. Despite its heuristic improvement of convergence, a rigorous mathematical justification for the benefits of Anderson mixing in RL has not yet been put forward. In this paper, we provide deeper insights into a class of acceleration schemes built on Anderson mixing that improve the convergence of deep RL algorithms. Our main results establish a connection between Anderson mixing and quasi-Newton methods and prove that Anderson mixing increases the convergence radius of policy iteration schemes by an extra contraction factor. The key focus of the analysis roots in the fixed-point iteration nature of RL. We further propose a stabilization strategy by introducing a stable regularization term in Anderson mixing and a differentiable, non-expansive MellowMax operator that can allow both faster convergence and more stable behavior. Extensive experiments demonstrate that our proposed method enhances the convergence, stability, and performance of RL algorithms.

show abstract

QUOTA: The Quantile Option Architecture for Reinforcement Learning

Cited by 3 publications

References 0 publications

Towards Safe, Explainable, and Regulated Autonomous Driving

Towards Safe, Explainable, and Regulated Autonomous Driving

Sample-based Distributional Policy Gradient

Damped Anderson Mixing for Deep Reinforcement Learning: Acceleration, Convergence, and Stabilization

Contact Info

Product

Resources

About