Multi-Task Multi-Agent Reinforcement Learning for Real-Time Scheduling of a Dual-Resource Flexible Job Shop with Robots

Abstract

In this paper, a real-time scheduling problem of a dual-resource flexible job shop with robots is studied. Multiple independent robots and their supervised machine sets form their own work cells. First, a mixed integer programming model is established, which considers the scheduling problems of jobs and machines in the work cells, and of jobs between work cells, based on the process plan flexibility. Second, in order to make real-time scheduling decisions, a framework of multi-task multi-agent reinforcement learning based on centralized training and decentralized execution is proposed. Each agent interacts with the environment and completes three decision-making tasks: job sequencing, machine selection, and process planning. In the process of centralized training, the value network is used to evaluate and optimize the policy network to achieve multi-agent cooperation, and the attention mechanism is introduced into the policy network to realize information sharing among multiple tasks. In the process of decentralized execution, each agent performs multiple task decisions through local observations according to the trained policy network. Then, observation, action, and reward are designed. Rewards include global and local rewards, which are decomposed into sub-rewards corresponding to tasks. The reinforcement learning training algorithm is designed based on a double-deep Q-network. Finally, the scheduling simulation environment is derived from benchmarks, and the experimental results show the effectiveness of the proposed method.

Software Reliability Model Estimation for an Indeterministic Crime Cluster through Reinforcement Learning

“…Next reward (Agreed/Silent) =M (next H). The crime type model is optimal for Hyperparameters with high crime rewards and silent low reward Hyperparameters, addressing the multi-label classification problem by Zhu et al [9]. Cross entropy can be utilized to enhance the probability of a model producing certain Hyperparameters to 1, indicating our preference for them.…”

Multi-Robot Task Planning for Efficient Battery Disassembly in Electric Vehicles

“…While this method could be extended to up to five robots, it was observed that it timed out in the case of six robots due to computational limitations. A study by Zhu et al focusing on the flexible job scheduling of multi-robot manipulators operating in a dynamic environment and the dual-resource FJSP (flexible job shop scheduling problem) was carried out [20]. The paper used a gate layer neural network feature that addresses multitask coordination.…”

Fusion Q-Learning Algorithm for Open Shop Scheduling Problem with AGVs

“…They validated the method for the workshop-scheduling problem, demonstrating excellent efficiency and scalability. Zhu et al [40] designed a dual deep Q-network method to solve problems considering intra-job and machine scheduling. They demonstrated the effectiveness of their method through comparisons with benchmark cases.…”