Use of Proximal Policy Optimization for the Joint Replenishment Problem

Vanvuchelen, Nathalie; Gijsbrechts, Joren; Boute, Robert

doi:10.1016/j.compind.2020.103239

Cited by 79 publications

(43 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[22], for example, use a single agent with complete information to dispatch production in a cloud manufacturing environment. [23] propose a way to solve the joint distribution problem: which items to select for which types of deliveries. In the logistics area, [24] show how to assign vehicles to platoons of trucks and where to send the platoons.…”

Section: Reinforcement Learning For Supply Chain Decision-makingmentioning

confidence: 99%

A Reinforcement Learning Powered Digital Twin to Support Supply Chain Decisions

Martin¹,

Oger²

2022

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

The complexity of making supply chain planning decisions is growing along with the Volatility, Uncertainty, Complexity and Ambiguity of supply chain environments. As a consequence, the complexity of designing adequate decision support systems is also increasing. New approaches emerged for supporting decisions, and digital twins is one of those. Concurrently, the artificial intelligence field is growing, including approaches such as reinforcement learning. This paper explores the potential of creating digital twins with reinforcement learning capabilities. It first proposes a framework for unifying digital twins and reinforcement learning into a single approach. It then illustrates how this framework is put into practice for making supply and delivery decisions within a drug supply chain use case. Finally, the results of the experiment are compared with results given by traditional approaches, showing the applicability of the proposed framework.

show abstract

Section: Reinforcement Learning For Supply Chain Decision-makingmentioning

confidence: 99%

A Reinforcement Learning Powered Digital Twin to Support Supply Chain Decisions

Martin¹,

Oger²

2022

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

show abstract

“…More empirical approaches were proposed following the aforementioned works, such as clipping the distance between the old and new policy in the proximal policy optimization (PPO) algorithm of Schulman, Wolski, Dhariwal, Radford, & Klimov (2017) . The PPO algorithm was successfully applied in the joint replenishment inventory problem by Vanvuchelen, Gijsbrechts, & Boute (2020) . The computational requirements could be further improved.…”

Section: Actor-critics and Other Hybrid Techniquesmentioning

confidence: 99%

“…Understand and improve performance of heuristic policies: The OR/OM community has a strong focus on knowledge building and benchmarking these new approaches against existing policies. Accordingly, most early studies that apply DRL to inventory management benchmark DRL performance against the state-of-theart problem-specific heuristic policies (see e.g., Gijsbrechts et al, 2019;Vanvuchelen et al, 2020 ). As DRL matures, we may reverse this process and use DRL to benchmark the performance of existing (or new) heuristic policies, and understand why-or whenthese heuristics perform better or worse.…”

Section: Blending Numerical and Analytical Approaches To Optimize Inventory Policiesmentioning

confidence: 99%

“…Better DRL training algorithms: Researchers in OR/OM may continue to draw from innovations within the active field of DRL. From Vanvuchelen et al (2020) , for instance, it appears that PPO requires less tuning compared to A3C as employed by Gijsbrechts et al (2019) , though it should be noted that the algorithms were not compared on the same problem. In particular, model-based algorithms may be tailored to inventory research with the option to generalize well across problems, just like MuZero learned to play multiple games at master class level ( Schrittwieser et al, 2020 ), or one may even develop DRL algorithms specifically tailored to the theoretical characteristics of inventory problems obtained after decades of in-depth research, as we discuss next.…”

Section: Scalability Of Drl Algorithmsmentioning

confidence: 99%

See 1 more Smart Citation

Deep Reinforcement Learning for Inventory Control: A Roadmap

et al. 2021

Self Cite

View full text Add to dashboard Cite

Deep reinforcement learning (DRL) has shown great potential for sequential decision-making, including early developments in inventory control. Yet, the abundance of choices that come with designing a DRL algorithm, combined with the intense computational effort to tune and evaluate each choice, may hamper their application in practice. This paper describes the key design choices of DRL algorithms to facilitate their implementation in inventory control. We also shed light on possible future research avenues that may elevate the current state-of-the-art of DRL applications for inventory control and broaden their scope by leveraging and improving on the structural policy insights within inventory research. Our discussion and roadmap may also spur future research in other domains within operations management.

show abstract

“…A variant of PPO algorithm called memory proximal policy optimization is presented to solve quantum control tasks [ 23 ]. In [ 24 ], a PPO-based machine learning algorithm is implemented to decide on the replenishments of a group of collaborating companies. However, to our best knowledge, PPO has never been resorted for AOR task.…”

Section: Related Workmentioning

confidence: 99%

Continuous Viewpoint Planning in Conjunction with Dynamic Exploration for Active Object Recognition

Sun

Kong

Wang

et al. 2021

Entropy

View full text Add to dashboard Cite

Active object recognition (AOR) aims at collecting additional information to improve recognition performance by purposefully adjusting the viewpoint of an agent. How to determine the next best viewpoint of the agent, i.e., viewpoint planning (VP), is a research focus. Most existing VP methods perform viewpoint exploration in the discrete viewpoint space, which have to sample viewpoint space and may bring in significant quantization error. To address this challenge, a continuous VP approach for AOR based on reinforcement learning is proposed. Specifically, we use two separate neural networks to model the VP policy as a parameterized Gaussian distribution and resort the proximal policy optimization framework to learn the policy. Furthermore, an adaptive entropy regularization based dynamic exploration scheme is presented to automatically adjust the viewpoint exploration ability in the learning process. To the end, experimental results on the public dataset GERMS well demonstrate the superiority of our proposed VP method.

show abstract

Use of Proximal Policy Optimization for the Joint Replenishment Problem

Cited by 79 publications

References 35 publications

A Reinforcement Learning Powered Digital Twin to Support Supply Chain Decisions

A Reinforcement Learning Powered Digital Twin to Support Supply Chain Decisions

Deep Reinforcement Learning for Inventory Control: A Roadmap

Continuous Viewpoint Planning in Conjunction with Dynamic Exploration for Active Object Recognition

Contact Info

Product

Resources

About