Deep Policy Dynamic Programming for Vehicle Routing Problems

Kool, Wouter; Hoof, Herke van; Gromicho, Joaquim; Welling, Max

doi:10.48550/arxiv.2102.11756

Cited by 15 publications

(29 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Joshi et al ( 2019) use a Graph Convolutional Network to construct TSP tours and show that by utilizing a parallelized beam search, auto-regressive construction approaches for the TSP can be outperformed. Kool et al (2021) extend the proposed model by Joshi et al (2019) for the CVRP while creating a hybrid approach that initiates partial solutions using a heatmap representation as a preprocessing step, before training a policy to create partial solutions and refining these through dynamic programming. Kaempfer & Wolf (2018) extend the learned heatmap approach to the number of tours to be constructed; their Permutation Invariant Pooling Network addresses the mTSP (a TSP involving multiple tours but no additional capacity constraints), where feasible solutions are obtained via a beam search and have been proven to outperform a meta-heuristic mTSP solver.…”

Section: Related Workmentioning

confidence: 99%

Section: D4 Evaluation On More Realistic Test Instancesmentioning

confidence: 99%

“…In this section, we provide generalization results for the test set provided in Kool et al (2021). We compare the LKH method and DPDP (Kool et al (2021)) to the generalization results that were achieved after training our model on uniformly-distributed data of graph size 50 only. Table 12 shows that, even though our approach is trained on uniformly distributed VRP50 data, it yields competitive results on the 10000 out of sample instances while being among the fastest methods.…”

Section: D4 Evaluation On More Realistic Test Instancesmentioning

confidence: 99%

“…Comparison of Score Normalization TechniquesB.3 GENERALIZATIONThis section documents how the supervised VRP models trained on graph sizes of 20, 50 and 100 generalize to the respective other problem sizes, as well as to more realistic problem instances (Uchoa100) published inKool et al (2021) that are generated for graph-sizes of 100 following the data distribution documented inUchoa et al (2017). The underlying dataset in Table6is the rejection-sampled version of the dataset used inKool et al (2019).…”

mentioning

confidence: 99%

“…Comparison to DPDP(Kool et al (2021)) and LKH for the VRP with 100 nodes on 10000 instances generated following the data generation process described inUchoa et al (2017). All values in the Table, except for "Ours", refer to the published results inKool et al (2021).…”

mentioning

confidence: 99%

See 4 more Smart Citations

Supervised Permutation Invariant Networks for Solving the CVRP with Bounded Fleet Size

Thyssens¹,

Falkner²,

Schmidt-Thieme³

2022

Preprint

View full text Add to dashboard Cite

Learning to solve combinatorial optimization problems, such as the vehicle routing problem, offers great computational advantages over classical operations research solvers and heuristics. The recently developed deep reinforcement learning approaches either improve an initially given solution iteratively or sequentially construct a set of individual tours. However, most of the existing learning-based approaches are not able to work for a fixed number of vehicles and thus bypass the complex assignment problem of the customers onto an apriori given number of available vehicles. On the other hand, this makes them less suitable for real applications, as many logistic service providers rely on solutions provided for a specific bounded fleet size and cannot accommodate short term changes to the number of vehicles. In contrast we propose a powerful supervised deep learning framework that constructs a complete tour plan from scratch while respecting an apriori fixed number of available vehicles. In combination with an efficient post-processing scheme, our supervised approach is not only much faster and easier to train but also achieves competitive results that incorporate the practical aspect of vehicle costs. In thorough controlled experiments we compare our method to multiple state-of-the-art approaches where we demonstrate stable performance, while utilizing less vehicles and shed some light on existent inconsistencies in the experimentation protocols of the related work.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: D4 Evaluation On More Realistic Test Instancesmentioning

confidence: 99%

Section: D4 Evaluation On More Realistic Test Instancesmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Supervised Permutation Invariant Networks for Solving the CVRP with Bounded Fleet Size

Thyssens¹,

Falkner²,

Schmidt-Thieme³

2022

Preprint

View full text Add to dashboard Cite

show abstract

A deep reinforcement learning framework with generalization performance for the large-scale capacitated vehicle routing problem

Yang

Lin

Chen

2023

Second International Conference on Statistics, Applied Mathematics, and Computing Science (CSAMCS 2022)

View full text Add to dashboard Cite

Combinatorial optimization has found its way into a variety of domains, including artificial intelligence and cybernetics. Deep Reinforcement Learning (DRL) has recently demonstrated its promise for developing heuristics for NP-hard routing problems. The current generalization performance of models needs to be improved, especially for large-scale routing problems. In this paper, we propose a hybrid approach for the Capacitated Vehicle Routing Problem (CVRP) based on DRL and adaptive large neighborhood search. The information representation of the neural network for CVRP is also improved by the combination of multi-head attention mechanism, pointer network and graph neural networks. The experimental results demonstrate that the optimization of our model on CVRP outperforms existing DRL techniques and some traditional algorithms. In addition, our method improves the training efficiency of the model and the performance of generalization to large-scale CVRP.

show abstract

Efficient Active Search for Combinatorial Optimization Problems

Hottung¹,

Kwon²,

Tierney³

2021

Preprint

View full text Add to dashboard Cite

Recently numerous machine learning based methods for combinatorial optimization problems have been proposed that learn to construct solutions in a sequential decision process via reinforcement learning. While these methods can be easily combined with search strategies like sampling and beam search, it is not straightforward to integrate them into a high-level search procedure offering strong search guidance. Bello et al. (2016) propose active search, which adjusts the weights of a (trained) model with respect to a single instance at test time using reinforcement learning. While active search is simple to implement, it is not competitive with state-of-the-art methods because adjusting all model weights for each test instance is very time and memory intensive. Instead of updating all model weights, we propose and evaluate three efficient active search strategies that only update a subset of parameters during the search. The proposed methods offer a simple way to significantly improve the search performance of a given model and outperform state-of-the-art machine learning based methods on combinatorial problems, even surpassing the well-known heuristic solver LKH3 on the capacitated vehicle routing problem. Finally, we show that (efficient) active search enables learned models to effectively solve instances that are much larger than those seen during training.Preprint. Under review.

show abstract

Deep Policy Dynamic Programming for Vehicle Routing Problems

Cited by 15 publications

References 38 publications

Supervised Permutation Invariant Networks for Solving the CVRP with Bounded Fleet Size

Supervised Permutation Invariant Networks for Solving the CVRP with Bounded Fleet Size

A deep reinforcement learning framework with generalization performance for the large-scale capacitated vehicle routing problem

Efficient Active Search for Combinatorial Optimization Problems

Contact Info

Product

Resources

About