Bayesian Recurrent Neural Network Models for Forecasting and Quantifying Uncertainty in Spatial-Temporal Data

McDermott, Patrick L.; Wikle, Christopher K.

doi:10.3390/e21020184

Cited by 87 publications

(43 citation statements)

References 67 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such models can be implemented in some contexts (e.g. Chatzis, 2015;Chien and Ku, 2016;Gan et al, 2016;McDermott and Wikle, 2017a) but are quite sensitive to particular data sets and are typically computationally prohibitive. More recently, approximate Bayesian methods such as variational Bayes (Tran et al, 2018), and scalable Bayesian methods (Snoek et al, 2015) have been used successfully in deep models.…”

Section: Combining the Dh-dstm And Dn-dstm Frameworkmentioning

confidence: 99%

See 1 more Smart Citation

Comparison of Deep Neural Networks and Deep Hierarchical Models for Spatio-Temporal Data

Wikle

2019

JABES

View full text Add to dashboard Cite

Spatio-temporal data are ubiquitous in the agricultural, ecological, and environmental sciences, and their study is important for understanding and predicting a wide variety of processes. One of the difficulties with modeling spatial processes that change in time is the complexity of the dependence structures that must describe how such a process varies, and the presence of high-dimensional complex datasets and large prediction domains. It is particularly challenging to specify parameterizations for nonlinear dynamic spatio-temporal models (DSTMs) that are simultaneously useful scientifically and efficient computationally. Statisticians have developed deep hierarchical models that can accommodate process complexity as well as the uncertainties in the predictions and inference. However, these models can be expensive and are typically application specific. On the other hand, the machine learning community has developed alternative "deep learning" approaches for nonlinear spatio-temporal modeling. These models are flexible yet are typically not implemented in a probabilistic framework. The two paradigms have many things in common and suggest hybrid approaches that can benefit from elements of each framework. This overview paper presents a brief introduction to the deep hierarchical DSTM (DH-DSTM) framework, and deep models in machine learning, culminating with the deep neural DSTM (DN-DSTM). Recent approaches that combine elements from DH-DSTMs and echo state network DN-DSTMs are presented as illustrations.

show abstract

Section: Combining the Dh-dstm And Dn-dstm Frameworkmentioning

confidence: 99%

“…McDermott and Wikle (2017b) made several modifications to the standard ESN model to account for a simple approach to uncertainty quantification in a spatio-temporal nonlinear forecasting setting. They considered a quadratic ESN model.…”

Section: An Ensemble Approachmentioning

confidence: 99%

Comparison of Deep Neural Networks and Deep Hierarchical Models for Spatio-Temporal Data

Wikle

2019

JABES

View full text Add to dashboard Cite

show abstract

“…Zhang et al [14] adopted LSTM to predict Sea Surface Temperature (SST). Mcdermott et al [15,16] made simple modifications to the basic RNN and applied it to a Lorenz simulation and two real-world nonlinear spatio-temporal forecasting applications. Moreover, they built a quadratic echo state network to do ensemble forecasting.…”

Section: Introductionmentioning

confidence: 99%

Prediction of North Atlantic Oscillation Index with Convolutional LSTM Based on Ensemble Empirical Mode Decomposition

Yuan

Luo

et al. 2019

Atmosphere

View full text Add to dashboard Cite

The North Atlantic Oscillation (NAO) is the most significant mode of the atmosphere in the North Atlantic, and it plays an important role in regulating the local weather and climate and even those of the entire Northern Hemisphere. Therefore, it is vital to predict NAO events. Since the NAO event can be quantified by the NAO index, an effective neural network model EEMD-ConvLSTM, which is based on Convolutional Long Short-Term Memory (ConvLSTM) with Ensemble Empirical Mode Decomposition (EEMD), is proposed for NAO index prediction in this paper. EEMD is applied to preprocess NAO index data, which are issued by the National Oceanic and Atmospheric Administration (NOAA), and NAO index data are decomposed into several Intrinsic Mode Functions (IMFs). After being filtered by the energy threshold, the remaining IMFs are used to reconstruct new NAO index data as the input of ConvLSTM. In order to evaluate the performance of EEMD-ConvLSTM, six methods were selected as the benchmark, which included traditional models, machine learning algorithms, and other deep neural networks. Furthermore, we forecast the NAO index with EEMD-ConvLSTM and the Rolling Forecast (RF) and compared the results with those of Global Forecast System (GFS) and the averaging of 11 Medium Range Forecast (MRF) model ensemble members (ENSM) provided by the NOAA Climate Prediction Center. The experimental results show that EEMD-ConvLSTM not only has the highest reliability from evaluation metrics, but also can better capture the variation trend of the NAO index data.

show abstract

“…There have also been Bayesian implementations of the ESN model (e.g., Chatzis, ; Li, Han, & Wang, ), but none of these has been implemented within a Markov chain Monte Carlo (MCMC) framework, as is the case here, where multiple levels of uncertainties can be accounted for. In the context of spatio‐temporal forecasting, McDermott and Wikle (, ) have shown that augmenting the traditional ESN with quadratic output terms (analogous to the quadratic nonlinear component in statistical DSTMs) and input embeddings (e.g., including lags of the input variables as motivated by Takens's () representation in dynamical systems) can improve forecast accuracy compared to traditional ESN models.…”

Section: Introductionmentioning

confidence: 99%

Deep echo state networks with uncertainty quantification for spatio‐temporal forecasting

McDermott

Wikle

2018

Environmetrics

View full text Add to dashboard Cite

Long‐lead forecasting for spatio‐temporal systems can entail complex nonlinear dynamics that are difficult to specify a priori. Current statistical methodologies for modeling these processes are often highly parameterized and, thus, challenging to implement from a computational perspective. One potential parsimonious solution to this problem is a method from the dynamical systems and engineering literature referred to as an echo state network (ESN). ESN models use reservoir computing to efficiently compute recurrent neural network forecasts. Moreover, multilevel (deep) hierarchical models have recently been shown to be successful at predicting high‐dimensional complex nonlinear processes, particularly those with multiple spatial and temporal scales of variability (such as those we often find in spatio‐temporal environmental data). Here, we introduce a deep ensemble ESN (D‐EESN) model. Despite the incorporation of a deep structure, the presented model is computationally efficient. We present two versions of this model for spatio‐temporal processes that produce forecasts and associated measures of uncertainty. The first approach utilizes a bootstrap ensemble framework, and the second is developed within a hierarchical Bayesian framework (BD‐EESN). This more general hierarchical Bayesian framework naturally accommodates non‐Gaussian data types and multiple levels of uncertainties. The methodology is first applied to a data set simulated from a novel non‐Gaussian multiscale Lorenz‐96 dynamical system simulation model and, then, to a long‐lead United States (U.S.) soil moisture forecasting application. Across both applications, the proposed methodology improves upon existing methods in terms of both forecast accuracy and quantifying uncertainty.

show abstract

Bayesian Recurrent Neural Network Models for Forecasting and Quantifying Uncertainty in Spatial-Temporal Data

Cited by 87 publications

References 67 publications

Comparison of Deep Neural Networks and Deep Hierarchical Models for Spatio-Temporal Data

Comparison of Deep Neural Networks and Deep Hierarchical Models for Spatio-Temporal Data

Prediction of North Atlantic Oscillation Index with Convolutional LSTM Based on Ensemble Empirical Mode Decomposition

Deep echo state networks with uncertainty quantification for spatio‐temporal forecasting

Contact Info

Product

Resources

About