Strategic capacity decision‐making in a stochastic manufacturing environment using real‐time approximate dynamic programming

Abstract

Abstract:In this study, we illustrate a real-time approximate dynamic programming (RTADP) method for solving multistage capacity decision problems in a stochastic manufacturing environment, by using an exemplary three-stage manufacturing system with recycle. The system is a moderate size queuing network, which experiences stochastic variations in demand and product yield. The dynamic capacity decision problem is formulated as a Markov decision process (MDP). The proposed RTADP method starts with a set of heuristics and learns a superior quality solution by interacting with the stochastic system via simulation. The curse-of-dimensionality associated with DP methods is alleviated by the adoption of several notions including "evolving set of relevant states," for which the value function table is built and updated, "adaptive action set" for keeping track of attractive action candidates, and "nonparametric k nearest neighbor averager" for value function approximation. The performance of the learned solution is evaluated against (1) an "ideal" solution derived using a mixed integer programming (MIP) formulation, which assumes full knowledge of future realized values of the stochastic variables (2) a myopic heuristic solution, and (3) a sample path based rolling horizon MIP solution. The policy learned through the RTADP method turned out to be superior to polices of 2 and 3.

A multi-stage stochastic programming approach for network capacity expansion with multiple sources of capacity

“…Atamtürk and Zhang , Mudchanatongsuk et al , Ordóñez and Zhao , and Karoonsoontawong are examples of addressing arc capacity expansion problem with robust optimization approach. Another technique dealing with stochastic capacity expansion is Markov Decision Processes as illustrated in Bean et al , Bhatnagar et al , and Pratikakis . However, to the best of our knowledge, there is no such work in stochastic network capacity expansion problems.…”

Reducing the computational effort of optimal process controllers for continuous state spaces by using incremental learning and post-decision state formulations

“…Global parametric function approximation can be realized with ANNs [5] or Evolutionary Value Function Approximations [15], combining Reinforcement Learning with Genetic Algorithms. In contrast, local nonparametric function approximation is represented by instance-based algorithms in terms of classical value tables with k-NN prediction [13,16] and kernel regression [5].…”

“…ADP has been successfully applied to process control problems [13,18,16] using time-independent value functions that are determined by Approximate Value Iteration. According to [13], local averagers such as k-NN predictors should be preferred over global approximators in the context of Approximate Value Iteration due to convergence issues.…”

Computationally efficient approximate dynamic programming for multi-site production capacity planning with uncertain demands

“…(Pratikakis et al 2006) developed a real-time adaptive dynamic programming method for planning and scheduling. (Pratikakis et al 2010) studied Approximate Dynamic Programming (ADP) method to address the curse-of-dimensionality associated with multistage capacity decision problems. More recently, (Lin et al 2014) developed a Stochastic Dynamic Programming (SDP) approach for multi-site capacity planning in thin-film transistor liquid crystal display (TFT-LCD) manufacturing considering demand uncertainty.…”