Synthesis-Style Pre-trained Auto-Correlation Transformer: A Zero-shot Learner on Long Ionospheric TEC Series Forecasting

Abstract

In this paper, we present a novel approach to improve the accuracy of TEC prediction through data augmentation. Prior works that adopt various deep-learning-based approaches suffer from two major problems. First, from a deep model perspective: LSTM models exhibit low performance on long-term data dependency, while self-attention-based methods ignore the temporal nature of time series, which results in an information utilization bottleneck. Second, the existing TEC actual data is limited and existing generative models fail to generate sufficient high-quality datasets. Our work leverages a two-stage deep learning framework for TEC prediction, stage 1: a time series generative model synthesis of sufficient data close to real data distribution, and stage 2: an Anto-correlation-based transformer to model temporal dependencies by presenting series-wise connections. Experiment on the 2018 TEC testing benchmark demonstrates that our method improves the accuracy by a large margin. The models trained on synthetic data had a notably lower RMSE of 1.17 TECU, while the RMSE for the IRI2016 model was 2.88 TECU. Our results show that the model significantly reduces monthly RMSE, displaying higher reliability in mid, high, low latitudes. Our model shows higher reliability and significantly reduces monthly RMSE and latitude RMSE. However, although our model performs better than IRI2016, low latitudes RMSE needs improvement, as values are generally above 2.5 TECU. This finding has important implications for the development of advanced TEC prediction models and highlights the potential of transformer models trained on synthetic data for a range of applications in ionospheric research and satellite communication systems.

Deep Learning‐Based Prediction of Global Ionospheric TEC During Storm Periods: Mixed CNN‐BiLSTM Method

“…Moreover, some studies have utilized Convolutional Long Short‐Term Memory neural networks, which incorporate spatial information into LSTM, to predict ionospheric parameters (Liu et al., 2022; Luo et al., 2023; Xia et al., 2022). Some studies used the Transformer to predict TEC, and by increasing the depth of the model, the prediction accuracy of the model was further improved (Shih et al., 2024; Yuan et al., 2023). In addition, nowcasting technique has been improved by Chen et al.…”