LGEANet: LSTM‐global temporal convolution‐external attention network for respiratory motion prediction

Abstract

To develop a deep learning network that treats the three-dimensional respiratory motion signals as a whole and considers the inter-dimensional correlation between signals of different directions for accurate respiratory tumor motion prediction. Methods: We propose a deep learning framework, named as LSTM-Global Temporal Convolution-External Attention Network (LGEANet). In LGEANet, we first feed each of the univariate time series into the Long Short-Term Memory (LSTM) module respectively and utilize the strength of the global temporal convolutional layer to discover the temporal pattern of the univariate signals from hidden states of the LSTM. Then, External attention is adopted to capture the dynamic dependence of the multiple time series. Also, a traditional autoregressive linear model in parallel to the non-linear neural network part was integrated to mitigate the scale insensitivity of the networks. A total of 304 motion traces for 31 patients are acquired from a public dataset in the experiments and four representative cases were selected for model evaluation. The respiratory signals were sampled at intervals of about 37.5 ms (26 frames per second) for an average duration of 71 min. Results: The proposed LGEANet achieved better performance with higher empirical correlation coefficient value (CORRs) and lower mean absolute error value (MAEs) and relative squared error value (RSEs) than other investigated models. For the four representative datasets, when the response time is less than 231 ms, the model can achieve CORRs more than 0.96. And the averaged position error reduction by using the proposed model was about 67% in the superior-inferior (SI) direction, 41% in the anterior-posterior (AP) direction and 38% in the right-left (RL) direction compared to that without prediction. The proposed network achieved the greatest error reduction in the SI direction, which is the main direction of tumor motion. Conclusions:The LGEANet achieves promising performance in minimizing the prediction error due to system latencies during real-time tumor motion tracking.

Prediction of the position of external markers using a recurrent neural network trained with unbiased online recurrent optimization for safe lung cancer radiotherapy

“…These networks led to higher RMSEs (0.9mm for the LSTM and 0.68mm for the TCN) than that of UORO at 30Hz in our study, equal to 0.40mm, despite relatively shorter response times (280ms for the LSTM and 400ms for the TCN) and the similarity between the signal amplitude in [59] (from 0.6mm to 51.2mm) and our study (from 6mm to 40mm). Likewise, the nRMSE corresponding to prediction at f = 26Hz with an architecture combining LSTMs, TCNs, external attention, and a linear autoregressive model in [60], equal to 0.31, was approximately 3 times higher than that of UORO at f = 30Hz, despite the low horizon h = 231ms in that work. Similarly, when forecasting 26Hz Cyberknife data with a network comprised of a transformer encoder and LSTM layers, Tan et al reported MAEs and RMSEs at h ≥ 400ms higher than those corresponding to UORO at f = 30Hz in our research [51].…”

“…Most previous works about respiratory motion forecasting focused on the prediction of one-dimensional (1D) respiratory signals, but considering the correlation between time series corresponding to different moving points and directions may improve general tumor po-sition estimation accuracy. A straightforward approach consists of concatenating these components into a single vector used as the network input [36,37]; some other studies use a specialized module to capture interdimensional information, such as external attention in [60]. It was reported in [19] that using principal components from successive 3D tumor centroïd positions as input led to a higher forecasting accuracy than performing coordinate-wise prediction when h ≥ 0.4s with several classical machine learning algorithms.…”

“…These shortfalls motivate further research on RNNs for respiratory motion forecasting. Furthermore, some recent works integrating recurrent and attention-based modules in the same architecture demonstrated high performance in respiratory motion prediction [51,60], further suggesting that research about RNNs is still impactful in that field.…”

“…Some datasets comprise more data than others and are publicly available, which indicates potentially more generalizable results. This is, for example, the case of the Cyberknife data from Georgetown University [7] used for instance in [51] and [60] among the studies in Table 6. In addition, the performance metrics reported may be defined differently.…”

“…In addition, the performance metrics reported may be defined differently. For instance, normalization by the amplitude and standard deviation of the signal is used in [27] and [60], respectively, to compute the nRMSE. Moreover, some previous studies reported metrics using data whose amplitude was rescaled from -1 to 1 [26,47] 21 .…”