Deep reinforcement learning based AGVs real-time scheduling with mixed rule for flexible shop floor in industry 4.0

Hu, Hao; Jia, Xiaofeng; He, Qixuan; Fu, Shifeng; Li, Kuo

doi:10.1016/j.cie.2020.106749

Cited by 131 publications

(35 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hu et al [30] transformed the real-time scheduling of the AGV (Automatic Guided Vehicle) into an MDP for resource handling and scheduling of the flexible jobshop of an AGV. They developed a DQN learning scheduling algorithm as a learning policy.…”

Section: Literature Reviewmentioning

confidence: 99%

Dynamic Jobshop Scheduling Algorithm Based on Deep Q Network

Zhao¹,

Wang

Tan

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Jobshop scheduling is a classic instance in the field of production scheduling. Solving and optimizing the scheduling process of the jobshop can greatly reduce the production cost of the workshop and improve the processing efficiency, thereby improving the market competitiveness of the manufacturing industry. A reasonable scheduling algorithm in jobshop can reduce manufacturing costs, shorten product delivery time, enhance customer service quality, and improve the overall performance of the manufacturing system. The jobshop scheduling problem is transformed into a reinforcement learning problem based on the Markov decision process. The performance of the adaptive scheduling algorithm in a dynamic manufacturing environment is improved based on the Deep Q Network (DQN). In the proposed scheduling algorithm, five state features of continuous value ranges are designed for input to a Deep Neural Network (DNN), as well as ten well-known heuristic dispatching rules are selected as the action set of the DQN. In the proposed scheduling algorithm, the target network and the prediction network are used to train the parameters. The DQN uses a "softmax" function as an action selection strategy during the training process, where it selects the dispatching rules with the greatest action value as the execution action. It resolves the problem whereby the sub-optimal action value is greater than the Q value of the optimal action in the early learning stage. Futhermore, the non-optimal action is selected with a greater probability in the later learning stage. Ten typical jobshop test instances called "LA" are used as a simulation object and operated in Python. The simulation results confirm that the proposed scheduling algorithm based on DQN has better performance and universality than a single dispatching rule or traditional Q learning algorithm.INDEX TERMS dynamic scheduling, deep Q network, deep reinforcement learning, dispatching rules, job shop scheduling.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Dynamic Jobshop Scheduling Algorithm Based on Deep Q Network

Zhao¹,

Wang

Tan

et al. 2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The non-interference constraint of double YCs and the consideration of multiple environmental variables are the research focus of these problems [7][8][9][10][11][12][13]. For AGV scheduling problem, most researchers focus on finding the optimal vehicle allocation and path planning [14,15].…”

Section: Literature Reviewmentioning

confidence: 99%

Hybrid Scheduling for Multi-Equipment at U-Shape Trafficked Automated Terminal Based on Chaos Particle Swarm Optimization

Yang

et al. 2021

JMSE

View full text Add to dashboard Cite

Aimed to improve the efficiency of port operations, Shanghai Zhenhua Heavy Industries Co., Ltd. (ZPMC) proposed a new U-shape trafficked automated terminal. The new U-shape trafficked automated terminal brings a new hybrid scheduling problem. A hybrid scheduling model for yard crane (YC), AGV and ET in the U-shape trafficked automated terminal yard is established to solve the problem. The AGV and ET yard lanes are assumed to be one-way lane. Take the YC, AGV and ET scheduling results (the container transportation sequences) as variables and the minimization of the maximum completion time as the objective function. A scheduling model architecture with hierarchical abstraction of scheduling objects is proposed to refine the problem. The total completion time is solved based on a static and dynamic mixed scheduling strategy. A chaotic particle swarm optimization algorithm with speed control (CCPSO) is proposed, which include a chaotic particle strategy, a particle iterative speed control strategy, and a particle mapping space for hybrid scheduling. The presented model and algorithm were applied to experiments with different numbers of containers and AGVs. The parameters of simulation part refer to Qinzhou Port. The simulation results show that CCPSO can obtain a near-optimal solution in a shorter time and find a better solution when the solution time is sufficient, comparing with the traditional particle swarm optimization algorithm, the adaptive particle swarm optimization algorithm and the random position particle swarm optimization algorithm.

show abstract

“…That is, the lack of knowledge support among these approaches prevents applying DT in assembly line design (Boje et al , 2020). With these observations, based on previous research about knowledge capture (Chen and Jia, 2019), knowledge representation (Chen and Jia, 2020) and knowledge-based intelligent skills (Hu et al , 2020), this paper introduces combining knowledge and DT toward AAL smart design, while analyzing its key enabling technologies including dynamic design knowledge library (DDK-Lib), knowledge-driven digital AAL rapid modeling and knowledge-based smart evaluation.…”

Section: Related Workmentioning

confidence: 99%

Knowledge-enabled digital twin for smart designing of aircraft assembly line

Chang

Jia

et al. 2021

Self Cite

View full text Add to dashboard Cite

Purpose The purpose of this paper is to provide a knowledge-enabled digital twin for smart design (KDT-SD) of aircraft assembly line (AAL) to enhance the AAL efficiency, performance and visibility. Modern AALs usually need to have capabilities such as digital-physical interaction and self-evaluation that brings significant challenges to traditional design method for AAL. The digital twin (DT) combining with reusable knowledge, as the key technologies in this framework, is introduced to promote the design process by configuring, understanding and evaluating design scheme. Design/methodology/approach The proposed KDT-SD framework is designed with the introduction of DT and knowledge. First, dynamic design knowledge library (DDK-Lib) is established which could support the various activities of DT in the entire design process. Then, the knowledge-driven digital AAL modeling method is proposed. At last, knowledge-based smart evaluation is used to understand and identify the design flaws, which could further improvement of the design scheme. Findings By means of the KDT-SD framework proposed, it is possible to apply DT to reduce the complexity and discover design flaws in AAL design. Moreover, the knowledge equips DT with the capacities of rapid modeling and smart evaluation that improve design efficiency and quality. Originality/value The proposed KDT-SD framework can provide efficient design of AAL and evaluate the design performance in advance so that the feasibility of design scheme can be improved as much as possible.

show abstract

Deep reinforcement learning based AGVs real-time scheduling with mixed rule for flexible shop floor in industry 4.0

Cited by 131 publications

References 40 publications

Dynamic Jobshop Scheduling Algorithm Based on Deep Q Network

Dynamic Jobshop Scheduling Algorithm Based on Deep Q Network

Hybrid Scheduling for Multi-Equipment at U-Shape Trafficked Automated Terminal Based on Chaos Particle Swarm Optimization

Knowledge-enabled digital twin for smart designing of aircraft assembly line

Contact Info

Product

Resources

About