Deep reinforcement learning‐based balancing and sequencing approach for mixed model assembly lines

Abstract

A multi-agent iterative optimisation method based on deep reinforcement learning is proposed for the balancing and sequencing problem in mixed model assembly lines. Based on the Markov decision process model for balancing and sequencing, a balancing agent using a deep deterministic policy gradient algorithm, a sequencing agent using an Actor-Critic algorithm, as well as an iterative interaction mechanism between these agents' output solutions are designed for realising the global optimisation of mixed model assembly lines. The exchange of solution information including assembly time and station workload in the iterative interaction realises the coordination of the worker assignment policy at the balancing stage and the production arrangement policy at the sequencing stage for the minimisation of work overload and idle time at stations. Through the comparative experiments with heuristic rules, genetic algorithms, and the original deep reinforcement learning algorithm, the effectiveness of the proposed method is demonstrated and discussed for small-scale instances as well as large-scale ones.This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

Load Balancing of Two-Sided Assembly Line Based on Deep Reinforcement Learning

“…In the area of AL balancing, Li et al [6] focused on the research of balancing AL in the digital domain, and designed a DRLA with the support of deep deterministic policy gradient (DDPG) to enhance the operation and simulation effect of the assembly line digital twin model. Lv et al [7] combined the sequencing problem with the assembly line balance problem and proposed a new version of DRLA on the basis of DDPG, in which an iterative interaction mechanism between task assembly time and station load were designed to achieve production task sequencing and worker allocation layer by layer. The objective was minimizing the work overload.…”

“…Ct(j, k) represents the completion time of station (j,k), and Ct max = max{Ct(j, k)} is the maximum thereof. Equation (7) indicates that any task can only be assigned to one station. Equations ( 8) and ( 9) show cycle time constraints-that is, the completion time of each station must be less than cycle time.…”

“…Moreover, the deep reinforcement learning algorithm has higher adaptability to the complex and ever-changing production environment, which is easy to adjust so as to alter the solution. In addition, although deep reinforcement learning algorithms have begun to be tried in solving such AL balancing problems [6,7], they are currently used to optimize simulation models and resource allocation. Therefore, this paper carries out the study on load balancing of TAL based on deep reinforcement learning for the first time.…”