Structure of optimal policies to periodic-review inventory models with convex costs and backorders for all values of discount factors

Naval Research Logistics

Lewis

2017

Self Cite

This paper studies convergence properties of optimal values and actions for discounted and averagecost Markov Decision Processes (MDPs) with weakly continuous transition probabilities and applies these properties to the stochastic periodic-review inventory control problem with backorders, positive setup costs, and convex holding/backordering costs. The following results are established for MDPs with possibly noncompact action sets and unbounded cost functions: (i) convergence of value iterations to optimal values for discounted problems with possibly non-zero terminal costs, (ii) convergence of optimal finite-horizon actions to optimal infinite-horizon actions for total discounted costs, as the time horizon tends to infinity, and (iii) convergence of optimal discount-cost actions to optimal average-cost actions for infinite-horizon problems, as the discount factor tends to 1.Being applied to the setup-cost inventory control problem, the general results on MDPs imply the optimality of (s, S) policies and convergence properties of optimal thresholds. In particular this paper analyzes the setup-cost inventory control problem without two assumptions often used in the literature: (a) the demand is either discrete or continuous or (b) the backordering cost is higher than the cost of backordered inventory if the amount of backordered inventory is large.

α \geq 0

for finite‐horizon problems and for all values of

α \in [0, 1)

for infinite‐horizon problems. In particular, the smallest possible values of the discount factor

α^{*}

mentioned in Theorem 6.10 are computed in Ref.…”

Section: Optimality Of (S S) Policies For Setup‐cost Inventory Contrmentioning

confidence: 99%

“…In particular, the smallest possible values of the discount factor

α^{*}

mentioned in Theorem 6.10 are computed in Ref. . Though the general theory of MDPs implies that the value functions

v_{t, α} (x),

G_{t, α} (x),

v_{α} (x),

and

G_{α} (x)

are lower semi‐continuous in x , it is proved in Ref.…”

Section: Optimality Of (S S) Policies For Setup‐cost Inventory Contrmentioning

confidence: 99%

“…Though the general theory of MDPs implies that the value functions

v_{t, α} (x),

G_{t, α} (x),

v_{α} (x),

and

G_{α} (x)

Section: Optimality Of (S S) Policies For Setup‐cost Inventory Contrmentioning

confidence: 99%

See 1 more Smart Citation

On the convergence of optimal actions for Markov decision processes and the optimality of (s, S) inventory policies

Naval Research Logistics

Lewis

2017

Self Cite

“…Therefore, (A.1) implies lim x→−∞ h(x) = +∞. As explained in Feinberg and Liang [12,Equations (2.3), (4.1)], the convexity of the function h imply that 1 + lim x→−∞ h(x)/cx < 1.…”

mentioning

confidence: 76%

“…We also need the additional assumption that E[h * (x − D)] < +∞ for all x ∈ X. Then the results in this paper formulated under Assumption 2 and the results in Feinberg and Lewis [11] and Feinberg and Liang [12,13] hold for the problems with the lead time L = 1, 2, . .…”

Section: )mentioning

confidence: 99%

Stochastic Setup-Cost Inventory Model With Backorders and Quasiconvex Cost Functions

Liang

2019

Prob. Eng. Inf. Sci.

Self Cite

This paper studies a periodic-review single-commodity setup-cost inventory model with backorders and holding/backlog costs satisfying quasiconvexity assumptions. We show that the Markov decision process for this inventory model satisfies the assumptions that lead to the validity of optimality equations for discounted and average-cost problems and to the existence of optimal (s, S) policies. In particular, we prove the equicontinuity of the family of discounted value functions and the convergence of optimal discounted lower thresholds to the optimal average-cost one for some sequences of discount factors converging to 1. If an arbitrary nonnegative amount of inventory can be ordered, we establish stronger convergence properties: (i) the optimal discounted lower thresholds s α converge to optimal average-cost lower threshold s; and (ii) the discounted relative value functions converge to average-cost relative value function. These convergence results previously were known only for subsequences of discount factors even for problems with convex holding/backlog costs. The results of this paper also hold for problems with fixed lead times.lower thresholds s α converge to optimal average-cost lower threshold s; and (ii) the discounted relative value functions converge to average-cost relative value function. These convergence results previously were known only for subsequences of discount factors even for problems with convex holding/backlog costs. The results of this paper hold for problems with deterministic positive lead times.For problems with convex holding/backlog cost functions, Scarf [21] introduced the concept of K-convexity to prove the optimality of (s, S) policies for finite-horizon problems with continuous demand and convex holding/backlog costs. Zabel [26] indicated some gaps in Scarf [21] and corrected them. References [2,3,4,5,6,11,12,13,18,21,23,25] deal with convex or linear holding/backlog cost functions. Iglehart [18] extended Scarf's [21] results to infinite-horizon problems with continuous demand. Veinott and Wagner [25] proved the optimality of (s, S) policies for both finite-horizon and infinite-horizon problems with discrete demand. Beyer and Sethi [3] completed the missing proofs in Iglehart [18] and Veinott and Wagner [25]. Chen and Simchi-Levi [4, 5] studied coordinating inventory control and pricing problems and proved the optimality of (s, S) policies without assuming that the demand is discrete or continuous. Under certain assumptions, their results imply the optimality of (s, S) policies for problems without pricing. Beyer et al. [2] and Huh et al. [17] studied problems with parameters depending on exogenous factors modeled by a Markov chain. Additional references can be found in monographs by Porteus [19] and Zipkin [28].The analysis of periodic-review inventory models is based on the theory of Markov Decision Processes (MDPs). However, most of inventory control papers use only basic facts from the MDP theory, and the corresponding general results had been unavailable for a long time. Fe...

Continuity of discounted values and the structure of optimal policies for periodic‐review inventory systems with setup costs

Naval Research Logistics

Kraemer

2023

This paper proves continuity of value functions in discounted periodic‐review single‐commodity total‐cost inventory control problems with continuous inventory levels, fixed ordering costs, possibly bounded inventory storage capacity, and possibly bounded order sizes for finite and infinite horizons. In each of these constrained models, the finite and infinite‐horizon value functions are continuous, there exist deterministic Markov optimal finite‐horizon policies, and there exist stationary deterministic Markov optimal infinite‐horizon policies. For models with bounded inventory storage and unbounded order sizes, this paper also characterizes the conditions under which ()st,St$$ \left({s}_t,{S}_t\right) $$ policies are optimal in the finite horizon and an (s,S)$$ \left(s,S\right) $$ policy is optimal in the infinite horizon.