Approximate dynamic programming for an inventory problem: Empirical comparison

Katanyukul, Tatpong; Duff, William S.; Chong, Edwin K. P.

doi:10.1016/j.cie.2011.01.007

Cited by 10 publications

(10 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A '*' marks an average aggregate cost and line beside it represents a 90% confidence interval, based on t-test statistics. On the y-axis, labels 'H', 'S', 'SL', 'RG', and 'DCB', and 'Roll' indicate results obtained from the 12-period Look-Ahead, Sarsa, Sarsa(λ), RG, and the DCB method, respectively 4 . The Roll-out method was used with perfect system information: Rollout simulation parameter values match those of the actual problem.…”

Section: Resultsmentioning

confidence: 99%

“…RBF has three sets of parameters: centers, scales, and weights. RBF centers and scales are set up asKatanyukul et al (2011). RBF weights are adjustable parameters whose values are determined by the method under investigated.…”

mentioning

confidence: 99%

“…RBF weights are adjustable parameters whose values are determined by the method under investigated. Variables ε t−1 and σ t−1 2 are excluded from state parameters for reasons discussed inKatanyukul et al (2011).…”

mentioning

confidence: 99%

See 2 more Smart Citations

Intelligent Inventory Control: Is Bootstrapping Worth Implementing?

Katanyukul

Chong

Duff

2012

Intelligent Information Processing VI

Self Cite

View full text Add to dashboard Cite

The common belief is that using Reinforcement Learning methods (RL) with bootstrapping gives better results than without. However, inclusion of bootstrapping increases the complexity of the RL implementation and requires significant effort. This study investigates whether inclusion of bootstrapping is worth the effort when applying RL to inventory problems. Specifically, we investigate bootstrapping of the temporal difference learning method by using eligibility trace. In addition, we develop a new bootstrapping extension to the Residual Gradient method to supplement our investigation. The results show questionable benefit of bootstrapping when applied to inventory problems. Significance tests could not confirm that bootstrapping had statistically significantly reduced costs of inventory controlled by a RL agent. Our empirical results are based on a variety of problem settings, including demand correlations, demand variances, and cost structures.

show abstract

Section: Resultsmentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Intelligent Inventory Control: Is Bootstrapping Worth Implementing?

Katanyukul

Chong

Duff

2012

Intelligent Information Processing VI

Self Cite

View full text Add to dashboard Cite

show abstract

“…More recently, Pratikakis (2008) suggested a real-time ADP approach based on nonparametric k-nearest neighbor scheme for the value function approximation; it was applied to a high dimensional light aromatic supply chain example. Other studies dealing with inventory problems through ADP are well listed in Katanyukul, Duff, and Chong (2011); however, the usefulness of ADP has not been fully explored in the process systems engineering community.…”

Section: Literature Reviewmentioning

confidence: 99%

Multi-time scale procurement planning considering multiple suppliers and uncertainty in supply and demand

Shin

Lee

2016

Computers & Chemical Engineering

View full text Add to dashboard Cite

“…Most of the ADP or NDP algorithms explained in those books and the review, however, are not for UMDP's but for finite-horizon or discounted MDP's. Katanyukul et al (2011) compared the learning-based and simulation-based approximate dynamic programming methods for finite-horizon inventory problems with demand forecasts and concluded that the simulation-based rollout method provides better performance. Desai et al (2012) proposed the smoothed approximate linear program for discounted MDP's and applied to the game of Tetris and a queuing network control problem.…”

Section: Introductionmentioning

confidence: 98%