A reinforcement learning approach for transaction scheduling in a shuttle‐based storage and retrieval system

Abstract

With recent Industry 4.0 developments, companies tend to automate their industries. Warehousing companies also take part in this trend. A shuttle‐based storage and retrieval system (SBS/RS) is an automated storage and retrieval system technology experiencing recent drastic market growth. This technology is mostly utilized in large distribution centers processing mini‐loads. With the recent increase in e‐commerce practices, fast delivery requirements with low volume orders have increased. SBS/RS provides ultrahigh‐speed load handling due to having an excess amount of shuttles in the system. However, not only the physical design of an automated warehousing technology but also the design of operational system policies would help with fast handling targets. In this work, in an effort to increase the performance of an SBS/RS, we apply a machine learning (ML) (i.e., Q‐learning) approach on a newly proposed tier‐to‐tier SBS/RS design, redesigned from a traditional tier‐captive SBS/RS. The novelty of this paper is twofold: First, we propose a novel SBS/RS design where shuttles can travel between tiers in the system; second, due to the complexity of operation of shuttles in that newly proposed design, we implement an ML‐based algorithm for transaction selection in that system. The ML‐based solution is compared with traditional scheduling approaches: first‐in‐first‐out and shortest process time (i.e., travel) scheduling rules. The results indicate that in most cases, the Q‐learning approach performs better than the two static scheduling approaches.

A Hybrid Simulation and Reinforcement Learning Algorithm for Enhancing Efficiency in Warehouse Operations

“…Numerical simulations demonstrated the algorithm's ability to handle a large number of orders simultaneously and to improve picking efficiency. Similarly, Y. Ekren and Arslan [45] presented an RL approach, specifically Q-learning, for transaction scheduling in a shuttlebased storage and retrieval system (SBS/RS). The proposed approach outperformed static scheduling approaches, demonstrating its effectiveness in improving the performance of SBS/RS.…”

Transaction Processing Policies in a Flexible Shuttle-based Storage and Re-trieval System by Real-time Data Tracking under Agent-based Modelling

“…Recent work is completed by Ekren and Arslan (2022) by applying a machine learning method to schedule transactions in a flexible SBSRS. However, due to the long training time of agents, this approach might prove costly for companies in application.…”

“…Recently, Ekren and Arslan (2022) and Arslan and Ekren (2022) have studied reinforcement and deep-Q learning modelling approaches in flexile SBSRSs, respectively. Turhanlar et al (2022) study flexible SBSRS for aisle-to-aisle design, where agent-based modelling is developed to prevent collision and deadlocks in the system.…”

“…To address this issue, a more flexible SBSRS design has been developed, where fewer shuttles can travel between multiple tiers within an aisle (Arslan and Ekren, 2022;Ekren and Arslan, 2022;Ekren et al, 2023). Figure 2 shows the new flexible SBSRS.…”

Autonomous mobile robot travel under deadlock and collision prevention algorithms by agent-based modelling in warehouses

“…SBS-RSs are widely utilised in warehouses and distribution centres to ensure improved system performance such as increased speed, storage density, accuracy, and throughput. Since a dedicated shuttle is assigned to each tier of an aisle, this automated warehousing technology is also referred to as tier-captive SBS-RS in the literature (Lerher, Ekren, Sari, et al 2015;Lerher, Ekren, Dukic, et al 2015;Ekren 2017;Ekren and Arslan 2022).…”