Transaction selection policy in tier-to-tier SBSRS by using Deep <i>Q</i>-Learning

Arslan, Bartu; Ekren, Banu Yetkin

doi:10.1080/00207543.2022.2148767

Cited by 12 publications

(8 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…DQL distinguishes itself by learning to make decisions. It does this by interacting with the environment and learning from the outcomes of its actions [33]. In an HS image, each pixel can be considered an agent that needs to be classified based on its spectral signature.…”

Section: Related Workmentioning

confidence: 99%

“…The agent's objective is to optimize the accumulation of these reward points. A comprehensive discussion of RL is presented in references [31][32][33]. The fundamental elements of RL include:…”

Section: Dqlmentioning

confidence: 99%

“…On the other hand, model-based strategies can efficiently use fewer samples but depend on having an accurate model of the environment, which may be challenging to achieve in intricate settings. In this paper, the DQL algorithm is employed [31][32][33].…”

Section: Dqlmentioning

confidence: 99%

See 2 more Smart Citations

Enhancing Hyper-Spectral Image Classification with Reinforcement Learning and Advanced Multi-Objective Binary Grey Wolf Optimization

Shoeibi,

Nevisi,

Salehi

et al. 2024

CMC

View full text Add to dashboard Cite

Hyperspectral (HS) image classification plays a crucial role in numerous areas including remote sensing (RS), agriculture, and the monitoring of the environment. Optimal band selection in HS images is crucial for improving the efficiency and accuracy of image classification. This process involves selecting the most informative spectral bands, which leads to a reduction in data volume. Focusing on these key bands also enhances the accuracy of classification algorithms, as redundant or irrelevant bands, which can introduce noise and lower model performance, are excluded. In this paper, we propose an approach for HS image classification using deep Q learning (DQL) and a novel multi-objective binary grey wolf optimizer (MOBGWO). We investigate the MOBGWO for optimal band selection to further enhance the accuracy of HS image classification. In the suggested MOBGWO, a new sigmoid function is introduced as a transfer function to modify the wolves' position. The primary objective of this classification is to reduce the number of bands while maximizing classification accuracy. To evaluate the effectiveness of our approach, we conducted experiments on publicly available HS image datasets, including Pavia University, Washington Mall, and Indian Pines datasets. We compared the performance of our proposed method with several state-of-the-art deep learning (DL) and machine learning (ML) algorithms, including long short-term memory (LSTM), deep neural network (DNN), recurrent neural network (RNN), support vector machine (SVM), and random forest (RF). Our experimental results demonstrate that the Hybrid MOBGWO-DQL significantly improves classification accuracy compared to traditional optimization and DL techniques. MOBGWO-DQL shows greater accuracy in classifying most categories in both datasets used. For the Indian Pine dataset, the MOBGWO-DQL architecture achieved a kappa coefficient (KC) of 97.68% and an overall accuracy (OA) of 94.32%. This was accompanied by the lowest root mean square error (RMSE) of 0.94, indicating very precise predictions with minimal error. In the case of the Pavia University dataset, the MOBGWO-DQL model demonstrated outstanding performance with the highest KC of 98.72% and an impressive OA of 96.01%. It also recorded the lowest RMSE at 0.63, reinforcing its accuracy in predictions. The results clearly demonstrate that the proposed MOBGWO-DQL architecture not only reaches a highly accurate model more quickly but also maintains superior performance throughout the training process.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Dqlmentioning

confidence: 99%

See 1 more Smart Citation

Enhancing Hyper-Spectral Image Classification with Reinforcement Learning and Advanced Multi-Objective Binary Grey Wolf Optimization

Shoeibi,

Nevisi,

Salehi

et al. 2024

CMC

View full text Add to dashboard Cite

show abstract

“…Arslan and Ekren (2021) apply deep Q ‐learning for a tier‐to‐tier SBS/RS for smart transaction selection of shuttles. By that, they aim to have system performance with decreased average cycle time per transaction performance metric.…”

Section: Literature Reviewmentioning

confidence: 99%

A reinforcement learning approach for transaction scheduling in a shuttle‐based storage and retrieval system

Ekren

Arslan

2022

Int Trans Operational Res

View full text Add to dashboard Cite

With recent Industry 4.0 developments, companies tend to automate their industries. Warehousing companies also take part in this trend. A shuttle‐based storage and retrieval system (SBS/RS) is an automated storage and retrieval system technology experiencing recent drastic market growth. This technology is mostly utilized in large distribution centers processing mini‐loads. With the recent increase in e‐commerce practices, fast delivery requirements with low volume orders have increased. SBS/RS provides ultrahigh‐speed load handling due to having an excess amount of shuttles in the system. However, not only the physical design of an automated warehousing technology but also the design of operational system policies would help with fast handling targets. In this work, in an effort to increase the performance of an SBS/RS, we apply a machine learning (ML) (i.e., Q‐learning) approach on a newly proposed tier‐to‐tier SBS/RS design, redesigned from a traditional tier‐captive SBS/RS. The novelty of this paper is twofold: First, we propose a novel SBS/RS design where shuttles can travel between tiers in the system; second, due to the complexity of operation of shuttles in that newly proposed design, we implement an ML‐based algorithm for transaction selection in that system. The ML‐based solution is compared with traditional scheduling approaches: first‐in‐first‐out and shortest process time (i.e., travel) scheduling rules. The results indicate that in most cases, the Q‐learning approach performs better than the two static scheduling approaches.

show abstract

“…To address this issue, a more flexible SBSRS design has been developed, where fewer shuttles can travel between multiple tiers within an aisle (Arslan and Ekren, 2022;Ekren and Arslan, 2022;Ekren et al, 2023). Figure 2 shows the new flexible SBSRS.…”

Section: Introductionmentioning

confidence: 99%

Transaction Processing Policies in a Flexible Shuttle-based Storage and Re-trieval System by Real-time Data Tracking under Agent-based Modelling

Y. Ekren

2023

Int. j. math. eng. manag. sci.

View full text Add to dashboard Cite

This study investigates priority assignment rules (PARs) for transaction processing in automated warehouses featuring a shuttle-based storage and retrieval system (SBSRS). By incorporating real-time data tracking through agent-based modeling, the research explores the unique aspect of the SBSRS design, which involves flexible travel of robotic order picker shuttles be-tween tiers. The paper proposes PARs under agent-based modeling to enhance multi-objective performance metrics, including average flow time (AFT), maximum flow time (MFT), outlier transaction AFT, and standard deviations of flow times (SD) within the system. Experimental evaluations are conducted with various warehouse designs, comparing the results against commonly used static scheduling rules. The findings demonstrate that real-time tracking policies significantly improve system performance. Specifically, prioritizing the processing of outliers based on transaction waiting time enhances MFT, SD, and other performance metrics, while minimizing adverse effects on AFT. Certain rules exhibit notable improvements in MFT and SD, while others achieve the lowest AFT values among all experiments. This paper contributes to the existing literature by presenting a multi-objective performance improvement procedure and highlighting the advantages of real-time data track-ing-based scheduling policies in automated warehousing systems.

show abstract

Transaction selection policy in tier-to-tier SBSRS by using Deep Q-Learning

Cited by 12 publications

References 43 publications

Enhancing Hyper-Spectral Image Classification with Reinforcement Learning and Advanced Multi-Objective Binary Grey Wolf Optimization

Enhancing Hyper-Spectral Image Classification with Reinforcement Learning and Advanced Multi-Objective Binary Grey Wolf Optimization

A reinforcement learning approach for transaction scheduling in a shuttle‐based storage and retrieval system

Transaction Processing Policies in a Flexible Shuttle-based Storage and Re-trieval System by Real-time Data Tracking under Agent-based Modelling

Contact Info

Product

Resources

About