Toward Breaking the Curse of Dimensionality: An FPTAS for Stochastic Dynamic Programs with Multidimensional Actions and Scalar States

Abstract

We propose a Fully Polynomial-Time Approximation Scheme (FPTAS) for stochastic dynamic programs with multidimensional action, scalar state, convex costs and linear state transition function. The action spaces are polyhedral and described by parametric linear programs. This type of problems finds applications in the area of optimal planning under uncertainty, and can be thought of as the problem of optimally managing a single non-discrete resource over a finite time horizon. We show that under a value oracle model for the cost functions this result for one-dimensional state space is "best possible", because a similar dynamic programming model with two-dimensional state space does not admit a PTAS.The FPTAS relies on the solution of polynomial-sized linear programs to recursively compute an approximation of the value function at each stage. Our paper enlarges the class of dynamic programs that admit an FPTAS by showing, under suitable conditions, how to deal with multidimensional action spaces and with vectors of continuous random variables with bounded support. These results bring us one step closer to overcoming the curse of dimensionality of dynamic programming.

Optimal Use and Replenishment of Two Substitutable Raw Materials in a Stochastic Capacitated Make-to-Order Production System

“…Nadarajah and Secomandi (2018) utilize the convexity of a class of stochastic dynamic program and generalize the least squares Monte Carlo method, which approximates the true value function with a weighted average of a collection of basis functions whose weights are determined by a least square regression based on estimates of value of certain states obtained by Monte Carlo simulation, to multi-dimensional state space cases. Finally, Halman and Nannicini (2019) leverage affine and convex properties in the objective function of the class of stochastic dynamic program they consider to develop a Finite Polynomial Time Approximation Scheme for stochastic dynamic programs with multidimensional actions and scalar states. While we show that our stochastic dynamic program has convex value functions, in contrast to the approach above which utilizes convexity to directly approximate the value function, our approach is novel in that we leverage the explicitly characterized structure of the optimal policy and discretize it to obtain an approximately optimal policy, and then use that to compute an approximation of the value function.…”

A Survey of Recent Progress in the Asymptotic Analysis of Inventory Systems

“…As noted in section 7.3.1, even in some cases where we know particular policies to be asymptotically optimal, we do not fully understand the computational complexity of implementing the relevant policies. This relates to the fact that a formal theory of computational complexity for the structured stochastic dynamic programs which arise in inventory control remains incomplete (as discussed in section 3.1), although we refer the interested reader to Halman et al (2009Halman et al ( , 2014, Halman and Nannicini (2019), and more generally Dyer and Stougie (2006), Shmoys and Swamy (2006), Papadimitriou and Tsitsiklis (1987), Sidford et al (2018) for relevant work on the complexity of stochastic dynamic programming. For the setting of ATO systems, DeValve et al (2020) have also made recent progress along these lines, where we note that relevant questions such as how often one must "re-solve" certain approximating optimization problems in online optimization is a question currently of high interest across multiple academic communities (see e.g., Vera and Banerjee (2019), Bumpensanti and Wang (2020)).…”

“…This relates to the fact that a formal theory of computational complexity for the structured stochastic dynamic programs which arise in inventory control remains incomplete (as discussed in section 3.1), although we refer the interested reader to Halman et al. (2009,2014), Halman and Nannicini (2019), and more generally Dyer and Stougie (2006), Shmoys and Swamy (2006), Papadimitriou and Tsitsiklis (1987), Sidford et al. (2018) for relevant work on the complexity of stochastic dynamic programming.…”

Quantum speedups for convex dynamic programming

“…where s ⋆ x denotes the optimizer in (17). Then, π ⋆ (x) − π(x) ≤ 4ε/µ gu , where π ⋆ (x) is the optimal action.…”

“…To solve DP problems, a possibility is to rely on the approximation techniques at the core of approximate dynamic programming (ADP) [6,28]. While ADP cannot resolve the curse of dimensionality in general, it can sometimes lead to a tractable solution approach under some, typically restrictive, conditions, see, e.g., [12,14,17].…”