Towards Long-Term Time-Series Forecasting: Feature, Pattern, and Distribution

Abstract

Long-term time-series forecasting (LTTF) has become a pressing demand in many applications, such as wind power supply planning. Transformer models have been adopted to deliver high prediction capacity because of the high computational self-attention mechanism. Though one could lower the complexity of Transformers by inducing the sparsity in point-wise self-attentions for LTTF, the limited information utilization prohibits the model from exploring the complex dependencies comprehensively. To this end, we propose an efficient Transformerbased model, named Conformer, which differentiates itself from existing methods for LTTF in three aspects: (i) an encoderdecoder architecture incorporating a linear complexity without sacrificing information utilization is proposed on top of slidingwindow attention and Stationary and Instant Recurrent Network (SIRN); (ii) a module derived from the normalizing flow is devised to further improve the information utilization by inferring the outputs with the latent variables in SIRN directly; (iii) the inter-series correlation and temporal dynamics in time-series data are modeled explicitly to fuel the downstream self-attention mechanism. Extensive experiments on seven real-world datasets demonstrate that Conformer outperforms the state-of-the-art methods on LTTF and generates reliable prediction results with uncertainty quantification.

Joint Sensing and Communication for Situational Awareness in Wireless THz Systems

“…6, the generalizability of our joint imputation and forecasting system is evaluated versus fluctuations and new unseen user behavior (which are new mobility patterns tuned in this testing period). We compare our approach to RNN-based and transformer-based approaches which are well known models for predicting temporal dynamics [37]. From Fig.…”

Solar Radiation Prediction Based on Conformer-GLaplace-SDAR Model

“…The Conformer model shows significantly improved performance and training speed due to innovations such as incorporating multi-variable and temporal dependencies, utilizing both stationary and instant recurrent network blocks, and implementing a hybrid convolutional module. The Conformer model developed by Yan Li et al [22] in 2023 efficiently and stably predicts long-period sequences with obvious periodicity in multivariate time-series data, addressing associated computational efficiency and stability issues. The Conformer model comprises three main components: the input representation block, the encoder-decoder architecture, and the normalizing flow block.…”

“…Based on multi-scale dynamics, a time-series frequently displays distinctive temporal patterns at different resolutions [22]. In order to extract the temporal patterns across various scales, this study extracted the four temporal frequencies of the year, month, week and day to establish a temporal resolution set T ⊆ {year, month, week, day}.…”

“…The convolution operation can identify temporal patterns within the data and assign weights to each variable's contribution towards these patterns. Specifically, it is defined as a function that utilizes a sliding window across the time series data [22], computing a weighted sum of values within each window, which is illustrated as follows:…”

“…The Conformer model enhances the recurrent network to improve information utilization without increasing time and memory complexity by leveraging RNN's ability to capture dynamic information through cycles in the node network. The trend and seasonal terms are extracted from the input sequence and integrated with local temporal patterns, forming the stationary and instant recurrent network [22]. This compensates for the global information loss caused by sliding-window attention while reducing time complexity.…”