Deep reinforcement learning for semiconductor production scheduling

Waschneck, Bernd; Reichstaller, André; Belzner, Lenz; Altenmüller, Thomas; Bauernhansl, Thomas; Knapp, Alexander; Kyek, Andreas

doi:10.1109/asmc.2018.8373191

Cited by 89 publications

(41 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…After that, they [65] modelled the real-time rescheduling task as a closed-loop control problem and trained a deep Q-network to select repair actions in response to unexpected events and disturbances. Waschneck et al [66] applied a deep Q-network to semiconductor manufacturing scheduling and trained a deep neural network with flexible user-defined objectives. Lin et al [67] proposed an edge computing-based smart manufacturing factory framework, based on which the DQN was adjusted to solve the JSP.…”

Section: Drl For Schedulingmentioning

confidence: 99%

“…To speed up model training, they also proposed a parallel training method which combined asynchronous updates with deep deterministic policy gradient. [34] Job shop scheduling S M Genetic algorithm [35] Dynamic job shop scheduling M M Genetic local search algorithm [36] Flexible manufacturing system S M Fuzzy rule-based system for an adaptive scheduling [37] Job shop scheduling S M Hybrid genetic algorithm [38] Dynamic flexible job shop scheduling M M Improved hybrid multi-phase quantum particle swarm algorithm [39] Dynamic flexible job shop scheduling S M Improved genetic algorithm [40] Dynamic job shop scheduling S M Four new proposed dispatching rules [41] Flexible job shop scheduling S M Game theory [42] Flexible job shop scheduling M M Improved multi-objective genetic algorithm [43] Distributed job shop scheduling S M hybrid ant colony algorithm combined with local search [44] Dynamic job shop scheduling S S Q-III [45,46] Job shop scheduling S S Q-learning to the single machine dispatching rule selection [47] Stochastic lot-scheduling S S Multi-agent RL approach [48] Stochastic production scheduling S M Homogeneous multi-agent system [ [66] Semiconductor production scheduling S M Deep Q-network [67] Job shop scheduling S M Deep Q-network [65] Socio-technical manufacturing system S M Deep Q-Network [68] Job shop scheduling S M Actor-Critic DRL M denotes multi, and S denotes single…”

Section: Drl For Schedulingmentioning

confidence: 99%

See 1 more Smart Citation

Research on Adaptive Job Shop Scheduling Problems Based on Dueling Double DQN

2020

View full text Add to dashboard Cite

show abstract

Section: Drl For Schedulingmentioning

confidence: 99%

Section: Drl For Schedulingmentioning

confidence: 99%

Research on Adaptive Job Shop Scheduling Problems Based on Dueling Double DQN

2020

View full text Add to dashboard Cite

show abstract

“…For a detailed overview of the latest RL research in the domain of production planning and control, we refer to the work of [9,10,20,22].…”

Section: Basics Of Rlmentioning

confidence: 99%

“…Hereinafter, a short overview of some of the most important settings and hyperparameters is given (see Table 2), based on a review of other DQN applications (e.g. [22]). A sequential replay memory is used with a limit of one million experiences.…”

Section: Rl Algorithm and Simulation Of Environmentmentioning

confidence: 99%

Reinforcement learning for an intelligent and autonomous production control of complex job-shops under time constraints

Altenmüller

Stüker

Waschneck

et al. 2020

Prod. Eng. Res. Devel.

Self Cite

View full text Add to dashboard Cite

Reinforcement learning (RL) offers promising opportunities to handle the ever-increasing complexity in managing modern production systems. We apply a Q-learning algorithm in combination with a process-based discrete-event simulation in order to train a self-learning, intelligent, and autonomous agent for the decision problem of order dispatching in a complex job shop with strict time constraints. For the first time, we combine RL in production control with strict time constraints. The simulation represents the characteristics of complex job shops typically found in semiconductor manufacturing. A real-world use case from a wafer fab is addressed with a developed and implemented framework. The performance of an RL approach and benchmark heuristics are compared. It is shown that RL can be successfully applied to manage order dispatching in a complex environment including time constraints. An RL-agent with a gain function rewarding the selection of the least critical order with respect to time-constraints beats heuristic rules strictly by picking the most critical lot first. Hence, this work demonstrates that a self-learning agent can successfully manage time constraints with the agent performing better than the traditional benchmark, a time-constraint heuristic combining due date deviations and a classical first-in-first-out approach.

show abstract

“…Regarding the resulting complex optimization problem, multi-agent reinforcement learning (MARL) provides high potential in this field because of its short reaction time and fair solution quality [13]. MARL has already been applied to both production control (e.g., [14][15][16]) and smart grid approaches to non-industrial applications (e.g., [17,18]).…”

Section: Introductionmentioning

confidence: 99%

Smart Grid for Industry Using Multi-Agent Reinforcement Learning

et al. 2020

View full text Add to dashboard Cite

The growing share of renewable power generation leads to increasingly fluctuating and generally rising electricity prices. This is a challenge for industrial companies. However, electricity expenses can be reduced by adapting the energy demand of production processes to the volatile prices on the markets. This approach depicts the new paradigm of energy flexibility to reduce electricity costs. At the same time, using electricity self-generation further offers possibilities for decreasing energy costs. In addition, energy flexibility can be gradually increased by on-site power storage, e.g., stationary batteries. As a consequence, both the electricity demand of the manufacturing system and the supply side, including battery storage, self-generation, and the energy market, need to be controlled in a holistic manner, thus resulting in a smart grid solution for industrial sites. This coordination represents a complex optimization problem, which additionally is highly stochastic due to unforeseen events like machine breakdowns, changing prices, or changing energy availability. This paper presents an approach to controlling a complex system of production resources, battery storage, electricity self-supply, and short-term market trading using multi-agent reinforcement learning (MARL). The results of a case study demonstrate that the developed system can outperform the rule-based reactive control strategy (RCS) frequently used. Although the metaheuristic benchmark based on simulated annealing performs better, MARL enables faster reactions because of the significantly lower computation costs for its own execution.

show abstract

Deep reinforcement learning for semiconductor production scheduling

Cited by 89 publications

References 11 publications

Research on Adaptive Job Shop Scheduling Problems Based on Dueling Double DQN

Research on Adaptive Job Shop Scheduling Problems Based on Dueling Double DQN

Reinforcement learning for an intelligent and autonomous production control of complex job-shops under time constraints

Smart Grid for Industry Using Multi-Agent Reinforcement Learning

Contact Info

Product

Resources

About