This proof-of-concept study provides a novel method for robust-stable scheduling in dynamic flow shops based on deep reinforcement learning (DRL) implemented with OpenAI frameworks. In realistic manufacturing environments, dynamic events endanger baseline schedules, which can require a cost intensive re-scheduling. Extensive research has been done on methods for generating proactive baseline schedules to absorb uncertainties in advance and in balancing the competing metrics of robustness and stability. Recent studies presented exact methods and heuristics based on Monte Carlo experiments (MCE), both of which are very computationally intensive. Furthermore, approaches based on surrogate measures were proposed, which do not explicitly consider uncertainties and robustness metrics. Surprisingly, DRL has not yet been scientifically investigated for generating robust-stable schedules in the proactive stage of production planning. The contribution of this article is a proposal on how DRL can be applied to manipulate operation slack times by stretching or compressing plan durations. The method is demonstrated using different flow shop instances with uncertain processing times, stochastic machine failures and uncertain repair times. Through a computational study, we found that DRL agents achieve about 98% result quality but only take about 2% of the time compared to traditional metaheuristics. This is a promising advantage for the use in real-time environments and supports the idea of improving proactive scheduling methods with machine learning based techniques.

This feasibility study utilized regression models to predict makespan robustness in dynamic production processes with uncertain processing times. Previous methods for robustness determination were computationally intensive (Monte Carlo experiments) or inaccurate (surrogate measures). However, calculating robustness efficiently is crucial for field-synchronous scheduling techniques. Regression models with multiple input features considering uncertain processing times on the critical path outperform traditional surrogate measures. Well-trained regression models internalize the behavior of a dynamic simulation and can quickly predict accurate robustness (correlation: r>0.98). The proposed method was successfully applied to a permutation flow shop scheduling problem, balancing makespan and robustness. Integrating regression models into a metaheuristic model, schedules could be generated that have a similar quality to using Monte Carlo experiments. These results suggest that employing machine learning techniques for robustness prediction could be a promising and efficient alternative to traditional approaches. This work is an addition to our previous extensive study about creating robust stable schedules based on deep reinforcement learning and is part of the applied research project, Predictive Scheduling.

No abstract