MARL-Based Cooperative Multi-AGV Control in Warehouse Systems

Abstract

Automated guided vehicles (AGVs) are essential components for the automation of fulfillment centers, a type of warehouse, where goods are stored on shelves and carried by AGVs. To increase the productivity in inventory management, a well-organized cooperative path control is required to transport goods to the designated picking stations. In this paper, we propose a QMIX-based scheme for the cooperative path control of multiple AGVs. Although QMIX is the one of popular cooperative multi-agent reinforcement learning algorithms, we find that QMIX alone was not enough to increase productivity in warehouse systems. So, we develop two novel techniques that can be used with QMIX: 1) sequential action masking that eliminates all the collision cases and 2) additional local loss that improves collaboration of individual AGVs. They help to encourage the AGVs to cooperate more for high productivity. By extensive simulations, we present the superiority of the proposed scheme on several layouts in fulfillment centers. The effect of cooperation among AGVs in the proposed scheme is verified through the comparison study with the existing algorithms. Additionally, we show the generalization performance by investigating the reusability of the model trained with the proposed scheme.

Optimal coordination in logistics warehousing with sensing and communication limits: A distributed differential game approach

“…The overall process of the AGVs handling parcels in the warehouse logistics environment is summarized as follows [1]: Task assignment: In the worktop, a worker sends the order to transport the parcels to the appropriate AGVs according to the specified demand; The collection of parcels: After receiving the assigned task, the AGVs drive from the starting position (charging station) to the designated shelf and lift the corresponding parcels; The transportation of parcels: When AGVs get all the designated parcels, they will transport the parcels to the corresponding sorting workers; Return to a charging station: When the AGVs are deactivated, they will move to the nearest charging stations. …”

“…The overall process of the AGVs handling parcels in the warehouse logistics environment is summarized as follows [1]:…”

“…With the development of industry and the increased level of automation, automated guided vehicles (AGVs), as core components of advanced logistics systems, have become popular in modern industrial logistics systems [1], due to their ability to reduce logistics transportation time, shorten the production cycle, and improve overall efficiency for the production line [2]. Recent studies reveal that an effective coordination policy, which considers objectives including efficient utilization of all AGV subsystems, respecting temporal and spatial constraints, and avoiding collisions, subject to sensing and communication limits, and changing environments, is essential in achieving the maximum transportation efficiency of the entire logistic warehousing system [3].…”

An Improved Bounded Conflict-Based Search for Multi-AGV Pathfinding in Automated Container Terminals

“…Li et al [9] proposed a novel quantum ant colony optimization algorithm that combines the advantages of various methods. Choi et al [10] proposed a QMIX-based scheme for cooperative path control of multiple AGVs. For automated container terminals, Guo et al [11] proposed an improved Dijkstra algorithm and an enhanced acceleration control method to solve the path planning problem for 42 AGVs.…”

Efficient Rigid Body Localization Based on Euclidean Distance Matrix Completion for AGV Positioning Under Harsh Environment

“…R EAL-time, continuous and high-accuracy positioning is paramount for the reliable operation of automatic guided vehicles (AGVs) widely used in modern ports, unmanned warehouses and intelligent factories [1]- [3]. Generally, AGV Fig.…”