Uninhibited Positional and Contextual Attention in Spectral-Based (SPT) Transformer with Multi-head Shortcut for Improved Remaining Useful Life Forecasting in Industry 4.0

Abstract

This research paper aims to forecast equipment's remaining useful life (RUL) to improve maintenance planning and reduce costs. This paper presents the spectral-based transformer (SPT) model, designed for predicting the remaining useful life (RUL) in the evolving maintenance landscape of industry 4.0. Proactive maintenance is becoming increasingly important as it improves performance and reduces losses. SPT utilizes advanced attention mechanisms and innovations, which have been evaluated on the C-MAPSS dataset to simulate various operations. The contributions include discrete cosine transform attention (DCTA), uninhibited positional and contextual attention (UPCA), multi-head shortcuts, and bidirectional structures. Component efficacy is rigorously assessed through ablations. The results demonstrate that SPT exhibits superior performance compared to other methods, with a notable advantage on the challenging FD002 and FD004 sub-datasets within the C-MAPSS dataset. The proposed method decreases the root mean square error (RMSE) by 14% and enhances the performance scores of FD002 and FD004 by 10% and 24%, respectively. Additionally, it reduces the RMSE of FD004 by 15%. The model outperforms current methods, showing stability and generalization across different subsets of data. SPT demonstrates proficiency in capturing degradation patterns, which shows the potential for accurate remaining useful life (RUL) prediction. This tool is designed for time-series regression and has potential applications in various industries. Future research could focus on expanding the system's capabilities to process higher frequencies and broader contexts effectively. The SPT method provides a thorough approach for predicting the remaining useful life (RUL). It can potentially improve maintenance decisions and system performance in the context of Industry 4.0.