Digital twin and deep reinforcement learning enabled real-time scheduling for complex product flexible shop-floor

Abstract

Real-time scheduling methods are essential and critical to complex product flexible shop-floor due to the dynamic events in the production process, such as new job insertions, machine breakdowns and frequent rework. Recently, digital twin (DT) technology can help identify disturbances by continuously comparing physical space with virtual space, which enables real-time scheduling and greatly reduces the deviation between pre-schedule and actual schedule. However, the conventional scheduling models and algorithms cannot satisfy the adaptiveness and timeliness requirements of optimization in DT enabled shop-floor (DTS). To address above challenges, an overall framework of DT enabled real-time scheduling (DTE-RS) for complex product shop-floor is proposed to effectively reduce adverse impacts of the dynamic disturbances and minimize the makespan. Firstly, complex product flexible job shop scheduling problem (CPFJSP) is formulated as Markov Decision Process (MDP), taking into account machine breakdown and new job insertions. Then, deep Q-network (DQN) based solution is developed to achieve optimal task dispatching according to real-time production state. Finally, the case study for aircraft overhaul shop-floor is conducted to demonstrate effectiveness and feasibility of the proposed real-time scheduling method. Through experimental comparison, it is indicated that the proposed method could effectively respond to dynamic disturbances and outperform the dynamic scheduling method in terms of makespan.

Dynamic scheduling for dual-objective job shop with machine breakdown by reinforcement learning

“…For a more accurate description of machine disturbances scenarios, the time when machine breakdown occurs and the repair time of the failed machines are defined as the uniform distribution in equations ( 5) and (6). The problem can be divided into four scenarios as presented in Table 3 as in literature.…”

“…[1][2][3][4] Traditionally, it is assumed that all resources in production are deterministic in JSSP. However, various random disturbances will inevitably occur in actual manufacturing, 5,6 resulting in reduced productivity and delayed deliveries, etc.…”

Research on Low-Carbon-Emission Scheduling of Workshop under Uncertainty

“…To solve the flexible job shop scheduling problem, Kacem et al [18] designed a hybrid Pareto algorithm based on fuzzy logic to minimize the maximum completion time and the total machine load. Zheng et al [19] developed a neighborhood search algorithm based on a multi-objective group to solve the fuzzy flexible job shop scheduling problem by minimizing the maximum fuzzy completion time and machine load. Chang et al [20] proposed an overall DT-enabled real-time scheduling (DTE-RS) framework for complex product shop-floors to effectively reduce adverse impacts of the dynamic disturbances and minimize the makespan based on a global twin.…”

A scheduling mechanism for hybrid flow shops with reworks under general queue time limits

“…Due to the practical applicability, hybrid flow shops can be found in various manufacturing systems such as semiconductors, display, glass, steel, food, chemicals, and aircraft maintenance. [1][2][3][4][5][6][7][8][9][10] There are a number of previous studies on hybrid flow shop scheduling, which can be classified by machine types (identical, uniform, or unrelated machines), job characteristics (precedence relations, sequence-dependent setups, machine eligibility, and queue time limits), and objectives (makespan, flow time, tardiness, and energy consumption). [11][12][13][14][15][16] Among them, this study focuses on queue time limit, which is especially important in wafer fabrication.…”

“…Due to the practical applicability, hybrid flow shops can be found in various manufacturing systems such as semiconductors, display, glass, steel, food, chemicals, and aircraft maintenance. 1–10…”