With the increasing dynamic nature of customer demand, production, product, and manufacturing design changes have become more frequent. Moreover, inadequate validation during the manufacturing design phase may result in additional issues, such as process redesign and layout reallocation, during the operation phase. Therefore, systems that can pre-validate and allow accurate and reliable analysis in the manufacturing design phase, as well as apply and optimize variations in production lines in real time, are required. Previously, digital twin (DT) has been studied a lot in product design and facility prognostics and management fields. Research on the system framework leading to DT utilization and optimization and analysis through DT in complex manufacturing systems with continuous processes such as production lines is insufficient. In this study, a system based on a DT and simulation results is developed; this system can reflect, analyze, and optimize dynamic changes in the design of processes and production lines in real time. First, the framework and application of the proposed system are designed. Subsequently, optimization methodologies based on heuristics and reinforcement learning (RL) are developed. Finally, the effectiveness and applicability of the proposed system are verified by implementing an actual DT application at a real manufacturing site.

Recently, manufacturing companies have been making efforts to increase resource utilization while ensuring the flexibility of production lines to respond to rapidly changing market environments and customer demand. In the high-tech manufacturing industry, which requires expensive manufacturing facilities and is capital-intensive, re-entrant production lines are used for efficient production with limited resources. In such a production system, a part visits a specific station repeatedly during the production period. However, a re-entrant production line requires an appropriate scheduling system because other parts with different processing requirements are processed at the same station. In this study, a re-entrant production line was modeled as a manufacturing environment via simulation, and an adaptive scheduling system was developed to improve its operational performance by applying deep reinforcement learning (DRL). To achieve this, a software architecture for integrating DRL with the simulation was developed and the states, actions, and rewards of the reinforcement learning (RL) agent were defined. Moreover, a discrete-event simulation control module was designed to collect data from the simulation model and evaluate the policy network trained via DRL. Finally, the applicability and effectiveness of the developed scheduling system were verified by conducting experiments on a hypothetical re-entrant production line.