Machine remaining life prediction based on multi-layer self-attention and temporal convolution network

Shang, Zhiwu; Zhang, Baoren; Li, Wanxiang; Qian, Shiqi; Zhang, Jie

doi:10.1007/s40747-021-00606-4

Cited by 17 publications

(10 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Self-attention mechanism [31][32][33] is an important improvement on traditional attention mechanism and plays a key role in neural networks. It aims to capture the internal correlations of data and can help the model focus more on informative information that makes a significant contribution to the output.…”

Section: Sm Unitmentioning

confidence: 99%

Cloud Computing Load Prediction in Double-channel Residual Self-attention Temporal Convolutional Network with Weight Adaptive Updating

Jiang

Guan

2023

Preprint

View full text Add to dashboard Cite

Resource load prediction is a prominent challenge issue with the widespread adoption of cloud computing. A novel cloud computing load prediction method has been proposed in Double-channel residual Self-attention Temporal convolutional Network with Weight adaptive updating (DSTNW). A Double-channel Temporal convolution Network model (DTN) has been developed. The double-channel dilated causal convolution has been adopted to replace a single channel dilated causal convolution in the DTN. A residual temporal self-attention mechanism has been proposed to improve the performance of network and focus on features with significant contributions from the DTN. Some errors for single and stacked Double-channel residual Self-attention Temporal convolutional Network (DSTN) have been evaluated. An adaptive weight strategy has been proposed to assign corresponding weights for the single and stacked DSTNs, respectively. Experimental results highlight that the developed method has outstanding prediction performance for cloud computing in comparison with some state-of-the-arts.

show abstract

Section: Sm Unitmentioning

confidence: 99%

Cloud Computing Load Prediction in Double-channel Residual Self-attention Temporal Convolutional Network with Weight Adaptive Updating

Jiang

Guan

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…To validate the superiority of the proposed method, the performance of the NT-TCN model is compared to other state-of-the-art models. In this part, some classic and latest methods including CNN [3], LSTM [9], DCNN [5], BiLSTM [15], MS-DCNN [7], ALSTM [18], AGCNN [1], Cap-LSTM [17], BiGRU-MMoE [13], MLSA-TCN [24] and our proposed NT-TCN are considered for comparison. Table 6 shows the results.…”

Section: Comparisons With Other Approachesmentioning

confidence: 99%

“…It performs better than standard LSTM and GRU on multiple tasks. Considering the contribution differences between different channels and different time steps, Shang et al [24] proposed an RUL prediction method based on multi-layer self-attention and TCN (MLSA-TCN). Although TCN shows great advantages in residual life prediction, it has not attracted enough attention.…”

Section: Introductionmentioning

confidence: 99%

Remaining useful life prediction combining temporal convolutional network with nonlinear target function

Liu¹,

Xu²,

Sun³

et al. 2022

Meas. Sci. Technol.

View full text Add to dashboard Cite

Remaining useful life prediction based on degraded data is the premise of intelligent maintenance of equipment. Since the degradation process is usually complex and changeable, the general deep learning prediction method brings large prediction deviation since using linear target function. In this paper, the temporal convolutional network combined with nonlinear target function (NT-TCN) is proposed to improve the prediction accuracy. The nonlinear target function is constructed by piecewise function to label degraded data. By means of dilated causal convolution and Mish activation function, the NT-TCN learn the time series characteristics of degeneration, thus high precision of life prediction is achieved. By analyzing NASA's C-MAPSS dataset, the prediction results show that the NT-TCN model has lower root mean squared error and evaluation score than the neural network model with the linear target function, it is also superior to some existing prediction methods.

show abstract

“…Self-attention models can effectively handle variablelength time series and establish long-range dependencies [32].…”

Section: International Journal Of Aerospace Engineeringmentioning

confidence: 99%

Aircraft Trajectory Prediction Based on Bayesian Optimised Temporal Convolutional Network–Bidirectional Gated Recurrent Unit Hybrid Neural Network

Huang

Ding

2022

International Journal of Aerospace Engineering

View full text Add to dashboard Cite

Efficient and accurate flight trajectory prediction is a key technology for promoting intelligent and informative air traffic management and improving the operational capabilities and predictability of air traffic. To address the problems in extracting hidden information from historical trajectory information, the approach must accurately select high-dimensional features related to the prediction target and overcome the short-term memory of the time series. Herein, we present a novel trajectory prediction model based on a dual-self-attentive (DSA)-temporal convolutional network (TCN)-bidirectional gated recurrent unit (BiGRU) neural network. In this model, the TCN provides highly stable training, high parallelism, and a flexible perceptual domain. The self-attentive mechanism of the TCN structure can focus on features that contribute the most to the output. After the TCN, the BiGRU network combined with the self-attentive mechanism is used to further bidirectionally mine the connections between the features and outputs of the trajectory sequence, and a Bayesian algorithm is used to optimise the hyperparameters of the model for optimal performance. A comparison and validation based on current well-known neural network models (i.e., CNN, TCN, GRU, and their variants) shows that the DSA-TCN-BiGRU model based on Bayesian hyperparameter optimisation has the best performance. Therefore, the improved predictive model is applicable and valuable, providing a basis for future decision trajectory-based operations.

show abstract

Machine remaining life prediction based on multi-layer self-attention and temporal convolution network

Cited by 17 publications

References 32 publications

Cloud Computing Load Prediction in Double-channel Residual Self-attention Temporal Convolutional Network with Weight Adaptive Updating

Cloud Computing Load Prediction in Double-channel Residual Self-attention Temporal Convolutional Network with Weight Adaptive Updating

Remaining useful life prediction combining temporal convolutional network with nonlinear target function

Aircraft Trajectory Prediction Based on Bayesian Optimised Temporal Convolutional Network–Bidirectional Gated Recurrent Unit Hybrid Neural Network

Contact Info

Product

Resources

About