Relative Value Function Approximation for the Capacitated Re-Entrant Line Scheduling Problem

Abstract

Abstract-The problem addressed in this work is that of determining how to allocate the workstation processing and buffering capacity in a capacitated re-entrant line to the job instances competing for it, in order to maximize its long-run / steady-state throughput, while maintaining the logical correctness of the underlying material flow, i.e., deadlock-free operations. An approximation scheme for the optimal policy that is based on Neuro-Dynamic Programming theory is proposed, and its performance is assessed through a numerical experiment. The derived results indicate that the proposed method holds considerable promise for providing a viable, computationally efficient approach to the problem, and highlight directions for further investigation.Note to Practitioners-Sequencing and scheduling problems arising in the context of contemporary manufacturing environments are known to be extremely hard. For this reason, in most practical situations, these problems have been resolved through the application of a number of heuristics -i.e., "rules of thumb" that are expected to provide reasonable performance. Things are complicated even further in the automated versions of these environments, since the applied sequencing and scheduling logic must guarantee, in addition to good performance, logically correct and smooth operation. Both, the logical and the performance-oriented control problem of flexibly automated production systems can be -and have been -addressed through formal systems theory. However, a challenging remaining problem is the approximation of the derived optimal policies in a way that will maintain nearoptimality, and at the same time, it will be computationally tractable in the context of the "real-world" applications. Our past work has addressed this approximation problem primarily with respect to the optimal logical control policy. The work presented in this paper undertakes the complementary problem of approximating the optimal scheduling policy. To this end, we employ some recently emerged results from a field known as NeuroDynamic Programming (essentially, a systematic approximation framework for Dynamic Programming). Our results indicate that the proposed approximation framework holds considerable promise towards developing a systematic analytical methodology for deriving near-optimal and logically correct scheduling policies for flexibly automated production systems. More specifically, it is shown that, when applied to some prototypical problems concerning the scheduling of re-entrant lines with finite buffering capacity at their workstations, the proposed approximation framework (i) effectively integrates past results concerning the logical control of these environments, and (ii) the obtained performance is consistently superior to the performance provided by the typically used heuristics.

Algebraic Deadlock Avoidance Policies for Sequential Resource Allocation Systems

“…Condition 3 introduces the "hold-while-waiting" effect in the considered resource allocation which is at the base of the considered Figure 4: An event-driven framework for the RAS Supervisory Control problem deadlock problems. 4 Condition 4 applies primarily to complex process flows that involve parallelization, and implies that the logic coordinating the execution of the various process threads does not "confound" enacted sub-processes belonging to different process instantiations. Finally, in order to facilitate the subsequent discussion on the complexity of the posed problems and the proposed solutions, we also introduce the quantity |Φ| ≡ |R| + | n j=1 S j | + m i=1 C i , which defines the size of the RAS Φ.…”

“…(iii) Finally, in Phase III, the set of admissible actions is provided to the performance-oriented component of the RAS supervisor in order to select the one that will be communicated eventually to the RAS environment, in a way that observes some performance considerations. In addition to this basic functionality, the RAS controller should be able to respond to the various contingencies taking place in the RAS domain, by (i) appropriately updating the RAS configuration database, and (ii) revising 4 As demonstrated by the examples presented in Figures 1 and 2, this "hold-while-waiting" effect frequently results from the need to physically buffer the various process instances at any single point in time, i.e., parts processed in a flexibly automated production system or vehicles in an AGV network are physical entities and they always need to be accommodated somewhere during their sojourn through the system. It must be noticed, however, that, while providing the necessary specificity for the underlying resource allocation dynamics, the aforestated assumptions do not compromise the modeling power of our framework, since one can capture any additional resource allocation dynamics by augmenting the specification of process Π j .…”

Performance optimization for a class of generalized stochastic Petri nets

“…1 From a more practical standpoint, the developments presented in this work have been motivated by the need to address the scheduling (especially the throughput maximization) problem of sequential resource allocation systems (RAS) that are controlled for deadlock-freedom, liveness and reversibility according to the supervisory control theory presented in [4], [5]. In [6], [4], it has been shown that this scheduling problem can be modeled as an average-reward Markov Decision Process (MDP), which is well-defined and effectively solvable due to the structural properties that are established by the aforementioned liveness-enforcing supervision (LES). On the other hand, the solvability of this MDP is practically limited by the state-space explosion that is typical for the considered RAS.…”

Computationally efficient neuro-dynamic programming approximation method for the capacitated re-entrant line scheduling problem

“…The literature contains two major approximation schemes for computing approximated value functions (Choi 2004). These are (i) the look-up table method (Bertsekas andRoy 1996), and (ii) parametric representation methods (Bertsekas and Tsitsiklis 1996, Tsitsiklis and Roy 1996, Roy 1998, Farias 2002, Choi and Reveliotis 2005. Look-up table methods essentially aggregate the underlying state space to a number of aggregated components on the basis of a preselected set of features maintaining one value for each aggregated component.…”

“…Of particular interest is the work that focuses on the average reward MDP problem for capacitated re-entrant lines (Choi and Reveliotis 2005). More specifically, the work of Choi and Reveliotis (2005) proposed a feature-based linear parametric approximation method that utilises a set of good feature functions in terms of (i) minimisation of some distance metric characterising the quality of the approximation and (ii) the corresponding greedy policy that tends to maximise throughput. Furthermore, this work showed that the proposed approximation framework effectively integrates past results relevant to the logical control of these environments by performing numerical experiments with some prototypical problems exemplifying the CRL scheduling problem.…”