Learning Temporal and Spatial Correlations Jointly: A Unified Framework for Wind Speed Prediction

Zhu, Qiaomu; Chen, Jinfu; Shi, Dongyuan; Zhu, Lin; Bai, Xiang; Duan, Xianzhong; Liu, Yilu

doi:10.1109/tste.2019.2897136

Cited by 160 publications

(63 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast, fixed moment MTS can be considered to be a frame of a video sequence-image. 33 Ideally, an 'image' is divided into i rows and j columns, which satisfy the following conditions.…”

Section: A New Methods For Pre-processing Of Mtsmentioning

confidence: 99%

A dual‐stage attention‐based Conv‐LSTM network for spatio‐temporal correlation and multivariate time series prediction

et al. 2021

View full text Add to dashboard Cite

Multivariate time series (MTS) prediction aims at predicting future time series by extracting multiple forms of dependencies of past time series. Traditional prediction methods and deep learning‐based prediction methods focus on extracting the dynamic relationships of certain aspects of MTS, especially the temporal characteristics, often neglecting the spatial and temporal dynamic correlations of MTS. Inspired by convolution neural network (CNN) and attention mechanism, this paper proposes a convolution LSTM network model based on MTS prediction with two‐stage attention. Specifically, we first propose a new MTS preprocessing method to perform convolution operations better. Then convolution layer extracts spatial correlation of MTS and LSTM model extracts temporal correlation. It is worth mentioning that the combination of attention mechanism and LSTM can effectively solve the problem of insufficient time dependency in MTS prediction. In addition, dual‐stage attention mechanism can effectively eliminate irrelevant information, select the relevant exogenous sequence, give it higher weight, and increase the past value of the target sequence to further eliminate irrelevant information. Finally, the MTS spatio‐temporal correlation is extracted to improve the prediction accuracy, and the model is interpreted. Experimental results show that the model has broad application prospects. Experiments based on typical datasets of finance, environment, and energy determine the optimal window size and hidden size of the prediction, and demonstrate that the model achieves the state‐of‐the‐art effect compared to the other four deep learning models. On top of that, the model is not only suitable for single‐step prediction of MTS, but also suitable for multistep prediction of time step in a certain range.

show abstract

Section: A New Methods For Pre-processing Of Mtsmentioning

confidence: 99%

A dual‐stage attention‐based Conv‐LSTM network for spatio‐temporal correlation and multivariate time series prediction

et al. 2021

View full text Add to dashboard Cite

show abstract

“…By doing so, I 2 DRNN is able to characterize multiscale information, so as to achieve more accurate prediction. Note that this information-theoretic analysis will not be affected by the operations in the input module, as the x t used for analysis, as shown in (6), is a general notation that represents the output from the input module.…”

Section: ) Structures (I 2 Drnn Versus Stacked Rnn)mentioning

confidence: 99%

“…One of the most challenging tasks in PSTA [1] is to learn the intrinsic dependence relationships among data, as, in many real-world applications, such as disease prediction [2], [3], climate forecast [4]- [6], and traffic prediction [7]- [10], such dependence relationships are generally exhibited at multiple spatiotemporal scales among heterogeneous data sources [11]- [13]. Taking infectious disease as an example, the infected case number in a specific region might be on a downward trend each year, but the actual case number for various smaller regions at a different time of year may fluctuate with a multitude of factors, such as environmental, geographic, meteorological, and demographic factors, at varying scales [3].…”

Section: Introductionmentioning

confidence: 99%

Demystifying Deep Learning in Predictive Spatiotemporal Analytics: An Information-Theoretic Framework

Tan

Liu

2021

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

Deep learning has achieved incredible success over the past years, especially in various challenging predictive spatiotemporal analytics (PSTA) tasks, such as disease prediction, climate forecast, and traffic prediction, where intrinsic dependence relationships among data exist and generally manifest at multiple spatiotemporal scales. However, given a specific PSTA task and the corresponding data set, how to appropriately determine the desired configuration of a deep learning model, theoretically analyze the model's learning behavior, and quantitatively characterize the model's learning capacity remains a mystery. In order to demystify the power of deep learning for PSTA in a theoretically sound and explainable way, in this article, we provide a comprehensive framework for deep learning model design and information-theoretic analysis. First, we develop and demonstrate a novel interactively and integratively connected deep recurrent neural network (I 2 DRNN) model. I 2 DRNN consists of three modules: an input module that integrates data from heterogeneous sources; a hidden module that captures the information at different scales while allowing the information to flow interactively between layers; and an output module that models the integrative effects of information from various hidden layers to generate the output predictions. Second, to theoretically prove that our designed model can learn multiscale spatiotemporal dependence in PSTA tasks, we provide an informationtheoretic analysis to examine the information-based learning capacity (i-CAP) of the proposed model. In so doing, we can tackle an important open question in deep learning, that is, how to determine the necessary and sufficient configurations of a designed deep learning model with respect to the given learning data sets. Third, to validate the I 2 DRNN model and confirm its i-CAP, we systematically conduct a series of experiments involving both synthetic data sets and real-world PSTA tasks. The experimental results show that the I 2 DRNN model outperforms both classical and state-of-the-art models on all data sets and PSTA tasks. More importantly, as readily validated, the proposed model captures the multiscale spatiotemporal dependence, which is meaningful in the real-world context. Furthermore, the model configuration that corresponds to the best performance on a given data set always falls into the range between the necessary and sufficient configurations, as derived from the informationtheoretic analysis.

show abstract

“…This approach has been used in a model named Predictive Deep Convolutional Neural Network (PDCNN) [13] where a CNN followed by an MLP was applied to forecast wind speeds in a farm of 100 wind turbines aligned in a rectangular grid and demonstrated superior results to classical machine learning techniques. Shortly after, the researchers introduced a follow-up model consisting of a CNN followed by a Long Short-Term Memory (LSTM) [14] RNN, called Predictive Spatio-Temporal Network (PSTN) [15], demonstrating increased performance compared to both the PDCNN and the LSTM alone.…”

Section: Geo-temporal Predictionmentioning

confidence: 99%

Localized Convolutional Neural Networks for Geospatial Wind Forecasting

2020

View full text Add to dashboard Cite

Convolutional Neural Networks (CNN) possess many positive qualities when it comes to spatial raster data. Translation invariance enables CNNs to detect features regardless of their position in the scene. However, in some domains, like geospatial, not all locations are exactly equal. In this work, we propose localized convolutional neural networks that enable convolutional architectures to learn local features in addition to the global ones. We investigate their instantiations in the form of learnable inputs, local weights, and a more general form. They can be added to any convolutional layers, easily end-to-end trained, introduce minimal additional complexity, and let CNNs retain most of their benefits to the extent that they are needed. In this work we address spatio-temporal prediction: test the effectiveness of our methods on a synthetic benchmark dataset and tackle three real-world wind prediction datasets. For one of them, we propose a method to spatially order the unordered data. We compare the recent state-of-the-art spatio-temporal prediction models on the same data. Models that use convolutional layers can be and are extended with our localizations. In all these cases our extensions improve the results, and thus often the state-of-the-art. We share all the code at a public repository.

show abstract

Learning Temporal and Spatial Correlations Jointly: A Unified Framework for Wind Speed Prediction

Cited by 160 publications

References 53 publications

A dual‐stage attention‐based Conv‐LSTM network for spatio‐temporal correlation and multivariate time series prediction

A dual‐stage attention‐based Conv‐LSTM network for spatio‐temporal correlation and multivariate time series prediction

Demystifying Deep Learning in Predictive Spatiotemporal Analytics: An Information-Theoretic Framework

Localized Convolutional Neural Networks for Geospatial Wind Forecasting

Contact Info

Product

Resources

About