Performance evaluation of decision making under uncertainty for low power heterogeneous platforms

Constantinescu, Denisa-Andreea; Navarro, Ángeles; Fernández‐Madrigal, Juan‐Antonio; Asenjo, Rafael

doi:10.1016/j.jpdc.2019.11.009

Cited by 1 publication

(1 citation statement)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…2 Related work Jóhannsson (2009) demonstrated that value iteration could be effectively run in parallel on a GPU soon after the introduction of CUDA. Subsequent research has evaluated the performance of GPU-accelerated value iteration on problems from economics and finance (Aamer et al 2020;Aldrich et al 2011;Duarte et al 2020;Kirkby 2017;Kirkby 2022) and route-finding and navigation (Chen and Lu 2013;Constantinescu et al 2020;Inamoto et al 2011;Ruiz and Hernández 2015). We have only identified a single study that applied this approach to an inventory control problem: Ortega et al (2019) implemented a custom value iteration algorithm in CUDA to find replenishment policies for a subset of perishable inventory problems originally described by Hendrix et al (2019).…”

Section: Introductionmentioning

confidence: 99%

Going faster to see further: GPU-accelerated value iteration and simulation for perishable inventory control using JAX

Farrington¹,

Li²,

Wong³

et al. 2023

Preprint

View full text Add to dashboard Cite

Value iteration can find the optimal replenishment policy for a perishable inventory problem, but is computationally demanding due to the large state spaces that are required to represent the age profile of stock. The parallel processing capabilities of modern GPUs can reduce the wall time required to run value iteration by updating many states simultaneously. The adoption of GPU-accelerated approaches has been limited in operational research relative to other fields like machine learning, in which new software frameworks have made GPU programming widely accessible. We used the Python library JAX to implement value iteration and simulators of the underlying Markov decision processes in a high-level API, and relied on this library's function transformations and compiler to efficiently utilize GPU hardware. Our method can extend use of value iteration to settings that were previously considered infeasible or impractical. We demonstrate this on example scenarios from three recent studies which include problems with over 16 million states and additional problem features, such as substitution between products, that increase computational complexity. We compare the performance of the optimal replenishment policies to heuristic policies, fitted using simulation optimization in JAX which allowed the parallel evaluation of multiple candidate policy parameters on thousands of simulated years. The heuristic policies gave a maximum optimality gap of 2.49%. Our general approach may be applicable to a wide range of problems in operational research that would benefit from large-scale parallel computation on consumer-grade GPU hardware.

show abstract