Generalized Value Iteration Networks:Life Beyond Lattices

Niu, Sufeng; Chen, Siheng; Guo, Hanyu; Targonski, Colin; Smith, Melinda; Kovačević, Jelena

doi:10.1609/aaai.v32i1.12081

Cited by 24 publications

(13 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Value Iteration Networks [54] and Gated Path Planning Networks [37] apply convolutional neural networks (CNN) on discrete maps, then predict the policy with a weighted attention sum over neighborhoods. Generalized Value Iteration Networks [43] and Velickovic et al…”

Section: Related Workmentioning

confidence: 99%

Reducing Collision Checking for Sampling-Based Motion Planning Using Graph Neural Networks

Yu¹,

Gao²

2022

Preprint

View full text Add to dashboard Cite

Sampling-based motion planning is a popular approach in robotics for finding paths in continuous configuration spaces. Checking collision with obstacles is the major computational bottleneck in this process. We propose new learning-based methods for reducing collision checking to accelerate motion planning by training graph neural networks (GNNs) that perform path exploration and path smoothing. Given random geometric graphs (RGGs) generated from batch sampling, the path exploration component iteratively predicts collision-free edges to prioritize their exploration. The path smoothing component then optimizes paths obtained from the exploration stage. The methods benefit from the ability of GNNs of capturing geometric patterns from RGGs through batch sampling and generalize better to unseen environments. Experimental results show that the learned components can significantly reduce collision checking and improve overall planning efficiency in challenging high-dimensional motion planning tasks.

show abstract

Section: Related Workmentioning

confidence: 99%

Reducing Collision Checking for Sampling-Based Motion Planning Using Graph Neural Networks

Yu¹,

Gao²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Due to the importance of the planning feature in NNs, various works extended form [24] and proposed new differentiable reward and transition functions that can stabilize learning in the network. For example, the authors in [14] proposes a novel convolution operator to learn and plan on spatial and irregular graphs.…”

Section: Value Iteration Network and Applicationsmentioning

confidence: 99%

Reward Shaping Using Convolutional Neural Network

Sami¹,

Otrok²,

Mourad³

et al. 2022

Preprint

View full text Add to dashboard Cite

In this paper, we propose Value Iteration Network for Reward Shaping (VIN-RS), a potential-based reward shaping mechanism using Convolutional Neural Network (CNN). The proposed VIN-RS embeds a CNN trained on computed labels using the message passing mechanism of the Hidden Markov Model. The CNN processes images or graphs of the environment to predict the shaping values. Recent work on reward shaping still has limitations towards training on a representation of the Markov Decision Process (MDP) and building an estimate of the transition matrix. The advantage of VIN-RS is to construct an effective potential function from an estimated MDP while automatically inferring the environment transition matrix. The proposed VIN-RS estimates the transition matrix through a self-learned convolution filter while extracting environment details from the input frames or sampled graphs. Due to (1) the previous success of using message passing for reward shaping; and (2) the CNN planning behavior, we use these messages to train the CNN of VIN-RS. Experiments are performed on tabular games, Atari 2600 and MuJoCo, for discrete and continuous action space. Our results illustrate promising improvements in the learning speed and maximum cumulative reward compared to the state-of-the-art.

show abstract

“…The Value Iteration Network (VIN) [1] is a CNN architecture to approximate several recursions of VI planning. It has been generalized to arbitrary graphs [2] and multiple levels of hierarchical planning [3]. The instability issues, especially with respect to long planning horizons, due to the recurrent structure are addressed in [4] and [5].…”

Section: Related Workmentioning

confidence: 99%

Hierarchies of Planning and Reinforcement Learning for Robot Navigation

Wöhlke

Schmitt

Hoof

2021

2021 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Solving robotic navigation tasks via reinforcement learning (RL) is challenging due to their sparse reward and long decision horizon nature. However, in many navigation tasks, high-level (HL) task representations, like a rough floor plan, are available. Previous work has demonstrated efficient learning by hierarchal approaches consisting of path planning in the HL representation and using sub-goals derived from the plan to guide the RL policy in the source task. However, these approaches usually neglect the complex dynamics and sub-optimal sub-goal-reaching capabilities of the robot during planning. This work overcomes these limitations by proposing a novel hierarchical framework that utilizes a trainable planning policy for the HL representation. Thereby robot capabilities and environment conditions can be learned utilizing collected rollout data. We specifically introduce a planning policy based on value iteration with a learned transition model (VI-RL). In simulated robotic navigation tasks, VI-RL results in consistent strong improvement over vanilla RL, is on par with vanilla hierarchal RL on single layouts but more broadly applicable to multiple layouts, and is on par with trainable HL path planning baselines except for a parking task with difficult non-holonomic dynamics where it shows marked improvements.

show abstract

Generalized Value Iteration Networks:Life Beyond Lattices

Cited by 24 publications

References 16 publications

Reducing Collision Checking for Sampling-Based Motion Planning Using Graph Neural Networks

Reducing Collision Checking for Sampling-Based Motion Planning Using Graph Neural Networks

Reward Shaping Using Convolutional Neural Network

Hierarchies of Planning and Reinforcement Learning for Robot Navigation

Contact Info

Product

Resources

About