Risk-Conditioned Distributional Soft Actor-Critic for Risk-Sensitive Navigation

Choi, Jin-Young; Dance, Christopher R.; Kim, Jung-Eun; Hwang, Seulbin; Park, Kyung-Sik

doi:10.1109/icra48506.2021.9560962

Cited by 17 publications

(4 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Risk-sensitive planning and control dates back to the 1970s, as exemplified by risk-sensitive Linear-Exponential-Quadratic-Gaussian [49,50] and risk-sensitive Markov Decision Processes (MDPs) [51]. More recent methods include risk-sensitive nonlinear MPC [6,52], Q-learning [44,53], and actor-critic [54,55] methods, for various types of risk measures. Refer to a recent survey [56] for further details.…”

Section: Related Workmentioning

confidence: 99%

RAP: Risk-Aware Prediction for Robust Planning

Nishimura¹,

Mercat²,

Wulfe³

et al. 2022

Preprint

View full text Add to dashboard Cite

Robust planning in interactive scenarios requires predicting the uncertain future to make risk-aware decisions. Unfortunately, due to long-tail safetycritical events, the risk is often under-estimated by finite-sampling approximations of probabilistic motion forecasts. This can lead to overconfident and unsafe robot behavior, even with robust planners. Instead of assuming full prediction coverage that robust planners require, we propose to make prediction itself risk-aware. We introduce a new prediction objective to learn a risk-biased distribution over trajectories, so that risk evaluation simplifies to an expected cost estimation under this biased distribution. This reduces the sample complexity of the risk estimation during online planning, which is needed for safe real-time performance. Evaluation results in a didactic simulation environment and on a real-world dataset demonstrate the effectiveness of our approach.

show abstract

Section: Related Workmentioning

confidence: 99%

RAP: Risk-Aware Prediction for Robust Planning

Nishimura¹,

Mercat²,

Wulfe³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…We aim to improve over both by providing a dynamic risk-adjustment method that is sensitive to change in both the policy and environment and thus broadly applicable. Choi et al [2021] employ a dynamic approach by conditioning the policy on the risk level α. Their method requires policy evaluation and training across different uniformly distributed α levels, which dramatically increases the number of observations and therefore the computational complexity needed for training.…”

Section: Related Workmentioning

confidence: 99%

“…The idea of ARA is based on the parametric uncertainty of the agent's predictions to adapt its risk-awareness, thus encouraging more cautious behaviour in states that have not been visited as often. As an additional advantage over prior work [Choi et al, 2021], it does not add significant overhead to training as it requires no additional episode evaluations.…”

Section: Introductionmentioning

confidence: 99%

Automatic Risk Adaptation in Distributional Reinforcement Learning

Schubert¹,

Eimer²,

Rosenhahn³

et al. 2021

Preprint

View full text Add to dashboard Cite

The use of Reinforcement Learning (RL) agents in practical applications requires the consideration of suboptimal outcomes, depending on the familiarity of the agent with its environment. This is especially important in safety-critical environments, where errors can lead to high costs or damage. In distributional RL, the risksensitivity can be controlled via different distortion measures of the estimated return distribution. However, these distortion functions require an estimate of the risk level, which is difficult to obtain and depends on the current state. In this work, we demonstrate the suboptimality of a static risk level estimation and propose a method to dynamically select risk levels at each environment step. Our method ARA (Automatic Risk Adaptation) estimates the appropriate risk level in both known and unknown environments using a Random Network Distillation error. We show reduced failure rates by up to a factor of 7 and improved generalization performance by up to 14% compared to both risk-aware and risk-agnostic agents in several locomotion environments.

show abstract

“…Compared to DRL, Distributional RL methods capture return distributions instead of only the expected return [12], and these methods have been used to train risk-sensitive agents for a variety of navigation tasks [34], [35], [36], [37], [14] in recent years. These works focus on the single robot navigation problem in environments with static obstacles or dynamic obstacles moving along pre-defined trajectories.…”

Section: Related Workmentioning

confidence: 99%

End-to-end Decentralized Multi-robot Navigation in Unknown Complex Environments via Deep Reinforcement Learning

Lin

Yang

Zheng

et al. 2019

2019 IEEE International Conference on Mechatronics and Automation (ICMA)

View full text Add to dashboard Cite

In this paper, a novel deep reinforcement learning (DRL)-based method is proposed to navigate the robot team through unknown complex environments, where the geometric centroid of the robot team aims to reach the goal position while avoiding collisions and maintaining connectivity. Decentralized robot-level policies are derived using a mechanism of centralized learning and decentralized executing. The proposed method can derive end-to-end policies, which map raw lidar measurements into velocity control commands of robots without the necessity of constructing obstacle maps. Simulation and indoor real-world unmanned ground vehicles (UGVs) experimental results verify the effectiveness of the proposed method.

show abstract

Risk-Conditioned Distributional Soft Actor-Critic for Risk-Sensitive Navigation

Cited by 17 publications

References 26 publications

RAP: Risk-Aware Prediction for Robust Planning

RAP: Risk-Aware Prediction for Robust Planning

Automatic Risk Adaptation in Distributional Reinforcement Learning

End-to-end Decentralized Multi-robot Navigation in Unknown Complex Environments via Deep Reinforcement Learning

Contact Info

Product

Resources

About