Q(λ) learning-based dynamic route guidance algorithm for overhead hoist transport systems in semiconductor fabs

Abstract

A learning-based dynamic routing algorithm is proposed for the overhead hoist transport (OHT) systems of semiconductor fabrication facilities (fabs). An OHT system, which consists of multiple vehicles moving at high speeds on guided rails, is the primary automated material-handling system (AMHS) in a fab. Modern large-scale fabs have hundreds of vehicles moving lots between multiple processing machines. The dynamic routing method is a route guidance method that dynamically selects the best vehicle paths under given traffic conditions and congestion levels. Building on the Q(λ) learning method, we develop a reinforcement learning-based dynamic routing algorithm called QLBWR(λ), which consists of a Boltzmann softmax policy and a reward function. The proposed algorithm uses real-time information to effectively guide each vehicle so that it avoids congestion and finds an efficient path. The algorithm is also designed with a low computational burden, such that the efficient route can be found for hundreds of vehicles in real time. Simulation analyses on an actual fab layout are used to compare the performance of the proposed algorithm with common static and dynamic algorithms. The results show that the proposed algorithm outperforms the benchmarking algorithms.

A machine learning approach for implementing data-driven production control policies

“…Machine learning has been used for controlling operations and processes in manufacturing (Monostori et al 1996, Irani et al 1993, Charest et al 2018, Cadavid et al 2020, Hwang and Jang 2020.…”

“…For using machine learning in more complicated settings, agent based models have been suggested (Aissani et al 2008, Monostori et al 2006. Reinforcement learning methods have been applied to control systems with degrading machines and managing the delivery of material in manufacturing settings (Paraschos et al 2020, Hwang andJang 2020). Reinforcement learning methods have the capacity to be applied to a large variety of problems.…”

Traffic Flow Speed Prediction in Overhead Transport Systems for Semiconductor Fabrication Using Dense-UNet

“…Efficient operation of overhead hoist transport (OHT) systems is important for the productivity of semiconductor processes [1]. In particular, it is important to predict traffic flow and congestion over time because OHT operations, such as dispatching [2][3][4] and routing [5][6][7][8], are highly dependent on traffic conditions. In this study, the p OHT congestion prediction issue is addressed based on volume data.…”

Production Flow Analysis in a Semiconductor Fab Using Machine Learning Techniques

“…A semiconductor fab operates continuously to produce wafer lots through a complex process. The high complexity comes from the re-entrances of the wafer lots into the same machines several times [1]. For effective operating of this complex system, an advanced technique is necessary, which allows us to capture the system's dynamics.…”

“…The measures were continuously obtained from the running system, and when the value became lower or higher than a certain threshold, some unusual system behaviors that can cause a reduction in the system's performance could be identified. For case (1), this situation triggered the simulation to be executed for finding better decisions. For case (2), new production decisions were made to deal with the updated situation.…”