Reinforcement learning for adaptive order dispatching in the semiconductor industry

Stricker, Nicole; Kuhnle, Andreas; Sturm, Roland; Friess, Simon

doi:10.1016/j.cirp.2018.04.041

Cited by 106 publications

(34 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…However, ML algorithms are not favorable for all industrial applications. The following properties are advantageous: (i) applications with a limited scope in terms of the number of states and actions (the learning period is dependent on these dimensions), (ii) responsive real-time decision systems (computing the output of a ML algorithm requires just linear operations), (iii) "cheap" training data (the trial-and-error approach is intensively data-driven) and (iv) complex environments that can hardly be described in detail (ability to generalize) [15]. This work brings the application of ML algorithms and the transition towards autonomous production systems one step closer to reality.…”

Section: Conclusion Discussion and Outlookmentioning

confidence: 99%

See 1 more Smart Citation

Application of Reinforcement Learning in Production Planning and Control of Cyber Physical Production Systems

Kuhnle¹,

Lanza²

2018

Machine Learning for Cyber Physical Systems

Self Cite

View full text Add to dashboard Cite

Cyber Physical Production Systems (CPPS) provide a huge amount and variety of process and production data. Simultaneously, operational decisions are getting ever more complex due to smaller batch sizes (down to batch size one), a larger product variety and complex processes in production systems. Production engineers struggle to utilize the recorded data to optimize production processes effectively. In contrast, CPPS promote decentralized decision-making, so-called intelligent agents that are able to gather data (via sensors), process these data, possibly in combination with other information via a connection to and exchange with others, and finally take decisions into action (via actors). Modular and decentralized decision-making systems are thereby able to handle far more complex systems than rigid and static architectures. This paper discusses possible applications of Machine Learning (ML) algorithms, in particular Reinforcement Learning (RL), and the potentials towards an production planning and control aiming for operational excellence.

show abstract

Section: Conclusion Discussion and Outlookmentioning

confidence: 99%

“…Moreover, the heuristic results show an almost stable performance that is not able to adapt to changing conditions. [15]…”

Section: Intelligent Order Dispatchingmentioning

confidence: 99%

Application of Reinforcement Learning in Production Planning and Control of Cyber Physical Production Systems

Kuhnle¹,

Lanza²

2018

Machine Learning for Cyber Physical Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…For a detailed overview of the latest RL research in the domain of production planning and control, we refer to the work of [9,10,20,22].…”

Section: Basics Of Rlmentioning

confidence: 99%

“…• Current machine where the action is asked for (10 entries, binary, one-hot coding for 10 work groups) • Loading status of all machines (10 entries, binary, idle or busy) • Product setup per machine (50 entries, binary, one-hot coding for 5 product variants and ten machines) • Product variant in the current machine's buffer slots (100 entries, binary, one-hot coding for up to twenty buffers and 5 product variants) • Order status per buffer slot (20 entries, real-value, 20 buffer slots) • Full or empty status per buffer slot (20 entries, realvalue, 20 buffer slots)…”

Section: State Space Modellingmentioning

confidence: 99%

Reinforcement learning for an intelligent and autonomous production control of complex job-shops under time constraints

Altenmüller

Stüker

Waschneck

et al. 2020

Prod. Eng. Res. Devel.

Self Cite

View full text Add to dashboard Cite

Reinforcement learning (RL) offers promising opportunities to handle the ever-increasing complexity in managing modern production systems. We apply a Q-learning algorithm in combination with a process-based discrete-event simulation in order to train a self-learning, intelligent, and autonomous agent for the decision problem of order dispatching in a complex job shop with strict time constraints. For the first time, we combine RL in production control with strict time constraints. The simulation represents the characteristics of complex job shops typically found in semiconductor manufacturing. A real-world use case from a wafer fab is addressed with a developed and implemented framework. The performance of an RL approach and benchmark heuristics are compared. It is shown that RL can be successfully applied to manage order dispatching in a complex environment including time constraints. An RL-agent with a gain function rewarding the selection of the least critical order with respect to time-constraints beats heuristic rules strictly by picking the most critical lot first. Hence, this work demonstrates that a self-learning agent can successfully manage time constraints with the agent performing better than the traditional benchmark, a time-constraint heuristic combining due date deviations and a classical first-in-first-out approach.

show abstract

“…Nonetheless, both papers do not consider the application of RL for the direct generation of job schedules. The first paper we found that describes RL agents being able to directly allocate and sequence jobs is from Stricker et al (2018). The different agent types are responsible for different production control decisions, such as selecting the next job to be processed or assigning the selected job to a machine [18].…”

Section: Related Workmentioning

confidence: 99%

Evolving Neural Networks to Solve a Two-Stage Hybrid Flow Shop Scheduling Problem with Family Setup Times

Lang¹,

Reggelin²,

Behrendt³

et al. 2020

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

We present a novel strategy to solve a two-stage hybrid flow shop scheduling problem with family setup times. The problem is derived from an industrial case. Our strategy involves the application of NeuroEvolution of Augmenting Topologies -a genetic algorithm, which generates arbitrary neural networks being able to estimate job sequences. The algorithm is coupled with a discrete-event simulation model, which evaluates different network configurations and provides training signals. We compare the performance and computational efficiency of the proposed concept with other solution approaches. Our investigations indicate that NeuroEvolution of Augmenting Topologies can possibly compete with state-of-the-art approaches in terms of solution quality and outperform them in terms of computational efficiency.

show abstract

Reinforcement learning for adaptive order dispatching in the semiconductor industry

Cited by 106 publications

References 10 publications

Application of Reinforcement Learning in Production Planning and Control of Cyber Physical Production Systems

Application of Reinforcement Learning in Production Planning and Control of Cyber Physical Production Systems

Reinforcement learning for an intelligent and autonomous production control of complex job-shops under time constraints

Evolving Neural Networks to Solve a Two-Stage Hybrid Flow Shop Scheduling Problem with Family Setup Times

Contact Info

Product

Resources

About