Learning to schedule (L2S): adaptive job shop scheduling using double deep Q network

Workneh, Abebaw Degu; Gmira, Maha

doi:10.1080/23080477.2023.2187528

Cited by 4 publications

(4 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(iii) It can be found that the DRL significantly outperforms all scheduling rules when trained on small-scale instances and generalized on large-scale instances, indicating that the method proposed in this study is effective when dealing with highdimensional input space; and for the whole learning process, DMU is the data used for testing, and it can be seen from the experimental data that the method proposed in this study can effectively learn to generate better for invisible instances solutions. (iv) Tested with the same parameters, the PPO algorithm [44] performs better on instances than DQN [41] and DDPG [58] and performs about the same as the metaheuristic on instances with a relatively small total number of JXMs but for larger instances, the performance of the method proposed in this study is significantly better. However, overall, regardless of the method used, the ability to solve large-scale problems is worse than the ability to solve small-scale problems, and the training error increases as the scale increases in comparison to DRL.…”

Section: Resultsmentioning

confidence: 85%

“…Liu [40] proposed an integrated architecture of DRL and MAS (DRL-MAS) to accomplish real-time scheduling in dynamic environments. Yang [41] developed a DDQN method to solve the scheduling problem of dynamic production lines. Luo [42] used DQN to integrate the dynamic, flexible job shop scheduling problem (FJSP) using DQN to minimize total latency and solve the problem of inserting new orders.…”

Section: Dynamic Job Shop Scheduling Based On Artificial Intelligence...mentioning

confidence: 99%

See 1 more Smart Citation

Multi-Agent Reinforcement Learning for Job Shop Scheduling in Dynamic Environments

Pu,

Li,

Rahimifard

2024

Sustainability

View full text Add to dashboard Cite

In response to the challenges of dynamic adaptability, real-time interactivity, and dynamic optimization posed by the application of existing deep reinforcement learning algorithms in solving complex scheduling problems, this study proposes a novel approach using graph neural networks and deep reinforcement learning to complete the task of job shop scheduling. A distributed multi-agent scheduling architecture (DMASA) is constructed to maximize global rewards, modeling the intelligent manufacturing job shop scheduling problem as a sequential decision problem represented by graphs and using a Graph Embedding–Heterogeneous Graph Neural Network (GE-HetGNN) to encode state nodes and map them to the optimal scheduling strategy, including machine matching and process selection strategies. Finally, an actor–critic architecture-based multi-agent proximal policy optimization algorithm is employed to train the network and optimize the decision-making process. Experimental results demonstrate that the proposed framework exhibits generalizability, outperforms commonly used scheduling rules and RL-based scheduling methods on benchmarks, shows better stability than single-agent scheduling architectures, and breaks through the instance-size constraint, making it suitable for large-scale problems. We verified the feasibility of our proposed method in a specific experimental environment. The experimental results demonstrate that our research can achieve formal modeling and mapping with specific physical processing workshops, which aligns more closely with real-world green scheduling issues and makes it easier for subsequent researchers to integrate algorithms with actual environments.

show abstract

Section: Resultsmentioning

confidence: 85%

Section: Dynamic Job Shop Scheduling Based On Artificial Intelligence...mentioning

confidence: 99%

Multi-Agent Reinforcement Learning for Job Shop Scheduling in Dynamic Environments

Pu,

Li,

Rahimifard

2024

Sustainability

View full text Add to dashboard Cite

show abstract

“…Lei et al (2022) [21] presented an end-to-end deep reinforcement framework to learn a policy that solves the FJSP with the use of a graph neural network in which multi-pointer graph networks (MPGNs) and a muti-PPO training algorithm are developed to learn two sub-policies, i.e., an operation action policy and a machine action policy. Abebaw et al (2023) [22] considered the JSSP as an iterative decisionmaking problem, and a DDQN is utilized for training the model and learning an optimal policy in which six continuous state features are formulated to record the production environment; an epsilon-greedy strategy is used on the action selection; furthermore, the reward and the penalty of the evaluation metric are designed. Zhang et al (2022) [23] used the PPO algorithm in the DRL framework to tackle the dynamic scheduling problem in a job shop manufacturing system with an unexpected event of the machine failure in which the transport agent is required to dispatch jobs/orders to machines then to sinks from machines after the task of jobs is completed.…”

Section: Related Workmentioning

confidence: 99%

“…The training process of the DDQN can be carried out with an exact number of machines and jobs, which is a simplified problem satisfying DRL. Although a simplified trained model does not apply to scenarios with different sizes of machines and jobs, variations such as the processing time, the number of a job's operations, available machines of an operation, and the randomness of job arrivals can be tolerant [22]. Another training way to generalize a model with the two variables included is to generate datasets of all possible production configurations and to use these benchmark examples during each training episode to train a convergent model.…”

Section: Conclusion and Future Research Potentialsmentioning

confidence: 99%

Efficient Multi-Objective Optimization on Dynamic Flexible Job Shop Scheduling Using Deep Reinforcement Learning Approach

Fan

Sun

et al. 2023

Processes

View full text Add to dashboard Cite

Previous research focuses on approaches of deep reinforcement learning (DRL) to optimize diverse types of the single-objective dynamic flexible job shop scheduling problem (DFJSP), e.g., energy consumption, earliness and tardiness penalty and machine utilization rate, which gain many improvements in terms of objective metrics in comparison with metaheuristic algorithms such as GA (genetic algorithm) and dispatching rules such as MRT (most remaining time first). However, single-objective optimization in the job shop floor cannot satisfy the requirements of modern smart manufacturing systems, and the multiple-objective DFJSP has become mainstream and the core of intelligent workshops. A complex production environment in a real-world factory causes scheduling entities to have sophisticated characteristics, e.g., a job’s non-uniform processing time, uncertainty of the operation number and restraint of the due time, avoidance of the single machine’s prolonged slack time as well as overweight load, which make a method of the combination of dispatching rules in DRL brought up to adapt to the manufacturing environment at different rescheduling points and accumulate maximum rewards for a global optimum. In our work, we apply the structure of a dual layer DDQN (DLDDQN) to solve the DFJSP in real time with new job arrivals, and two objectives are optimized simultaneously, i.e., the minimization of the delay time sum and makespan. The framework includes two layers (agents): the higher one is named as a goal selector, which utilizes DDQN as a function approximator for selecting one reward form from six proposed ones that embody the two optimization objectives, while the lower one, called an actuator, utilizes DDQN to decide on an optimal rule that has a maximum Q value. The generated benchmark instances trained in our framework converged perfectly, and the comparative experiments validated the superiority and generality of the proposed DLDDQN.

show abstract

Deep reinforcement learning-based dynamic scheduling for resilient and sustainable manufacturing: A systematic review

Zhang,

Juraschek,

Herrmann

2024

Journal of Manufacturing Systems

View full text Add to dashboard Cite

Learning to schedule (L2S): adaptive job shop scheduling using double deep Q network

Cited by 4 publications

References 44 publications

Multi-Agent Reinforcement Learning for Job Shop Scheduling in Dynamic Environments

Multi-Agent Reinforcement Learning for Job Shop Scheduling in Dynamic Environments

Efficient Multi-Objective Optimization on Dynamic Flexible Job Shop Scheduling Using Deep Reinforcement Learning Approach

Deep reinforcement learning-based dynamic scheduling for resilient and sustainable manufacturing: A systematic review

Contact Info

Product

Resources

About