Quantifying Uncertainty in Deep Spatiotemporal Forecasting

Wu, Dongxia; Gao, Liyao; Chinazzi, Matteo; Xiong, Xinyue; Vespignani, Alessandro; Ma, Yuanlin; Yu, Rose

doi:10.1145/3447548.3467325

Cited by 38 publications

(30 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [22,23,24] including the recent state-of-the-art UQ estimation work [25]. The default hyperparameter configuration used for DCRNN is as follows: batch size -64; filter type -dual/bidirectional random walk (captures both the upstream and the downstream traffic dynamic); number of diffusion steps -2; RNN layers -2; RNN units per layer-64; optimizer -Adam; threshold for gradient clipping -5; initial learning rate -0.01; and learning rate decay -0.1.…”

Section: Resultsmentioning

confidence: 99%

“…The DCRNN model uses the past 60 minutes of time series data at each node of the graph to forecast traffic for the next 5, 10, 15, ..., 60 minutes. To evaluate the performance, we use the same metrics used in the current state-of-the-art DCRNN UQ estimation work [25]. These metrics are the mean absolute error (MAE), mean interval score (MIS), and Interval score.…”

Section: Resultsmentioning

confidence: 99%

“…In this section we compare the uncertainty estimates from our proposed DCRNN-SQRUQ with several other DCRNN UQ techniques that were used in a recent study [25] and show that our method outperforms all other methods. For the experimental comparison, we adopt the same setup in terms of error metrics, forecasting horizons, and confidence interval as described in [25]. The uncertainty estimation techniques reported in [25] are as follows.…”

Section: Uncertainty Estimation and Comparison With The State-of-the-artmentioning

confidence: 99%

“…For the experimental comparison, we adopt the same setup in terms of error metrics, forecasting horizons, and confidence interval as described in [25]. The uncertainty estimation techniques reported in [25] are as follows. ( 1) Bootstrap [28]: This method randomly samples 50% of training data points to train DCRNN model.…”

Section: Uncertainty Estimation and Comparison With The State-of-the-artmentioning

confidence: 99%

“…(4) Mean Interval Score [25]: MIS is directly used in the loss function along with MAE, and a multiheaded model is used to jointly output the upper bound, lower bound, and prediction for a given input. ( 5) Monte Carlo Dropout [31]: This method uses the algorithm develops by Zhu et al [32].…”

Section: Uncertainty Estimation and Comparison With The State-of-the-artmentioning

confidence: 99%

See 4 more Smart Citations

Deep-Ensemble-Based Uncertainty Quantification in Spatiotemporal Graph Neural Networks for Traffic Forecasting

Mallick¹,

Balaprakash²,

Macfarlane³

2022

Preprint

View full text Add to dashboard Cite

Deep-learning-based data-driven forecasting methods have produced impressive results for traffic forecasting. A major limitation of these methods, however, is that they provide forecasts without estimates of uncertainty, which are critical for real-time deployments. We focus on a diffusion convolutional recurrent neural network (DCRNN), a state-of-the-art method for short-term traffic forecasting. We develop a scalable deep ensemble approach to quantify uncertainties for DCRNN. Our approach uses a scalable Bayesian optimization method to perform hyperparameter optimization, selects a set of high-performing configurations, fits a generative model to capture the joint distributions of the hyperparameter configurations, and trains an ensemble of models by sampling a new set of hyperparameter configurations from the generative model. We demonstrate the efficacy of the proposed methods by comparing them with other uncertainty estimation techniques. We show that our generic and scalable approach outperforms the current state-of-the-art Bayesian and number of other commonly used frequentist techniques.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Uncertainty Estimation and Comparison With The State-of-the-artmentioning

confidence: 99%

Section: Uncertainty Estimation and Comparison With The State-of-the-artmentioning

confidence: 99%

Section: Uncertainty Estimation and Comparison With The State-of-the-artmentioning

confidence: 99%

See 3 more Smart Citations

Deep-Ensemble-Based Uncertainty Quantification in Spatiotemporal Graph Neural Networks for Traffic Forecasting

Mallick¹,

Balaprakash²,

Macfarlane³

2022

Preprint

View full text Add to dashboard Cite

show abstract

A review of predictive uncertainty estimation with machine learning

Tyralis,

Papacharalampous

2024

Artif Intell Rev

View full text Add to dashboard Cite

Predictions and forecasts of machine learning models should take the form of probability distributions, aiming to increase the quantity of information communicated to end users. Although applications of probabilistic prediction and forecasting with machine learning models in academia and industry are becoming more frequent, related concepts and methods have not been formalized and structured under a holistic view of the entire field. Here, we review the topic of predictive uncertainty estimation with machine learning algorithms, as well as the related metrics (consistent scoring functions and proper scoring rules) for assessing probabilistic predictions. The review covers a time period spanning from the introduction of early statistical (linear regression and time series models, based on Bayesian statistics or quantile regression) to recent machine learning algorithms (including generalized additive models for location, scale and shape, random forests, boosting and deep learning algorithms) that are more flexible by nature. The review of the progress in the field, expedites our understanding on how to develop new algorithms tailored to users’ needs, since the latest advancements are based on some fundamental concepts applied to more complex algorithms. We conclude by classifying the material and discussing challenges that are becoming a hot topic of research.

show abstract

Signal propagation in complex networks

et al. 2023

Physics Reports

138

View full text Add to dashboard Cite

Quantifying Uncertainty in Deep Spatiotemporal Forecasting

Cited by 38 publications

References 16 publications

Deep-Ensemble-Based Uncertainty Quantification in Spatiotemporal Graph Neural Networks for Traffic Forecasting

Deep-Ensemble-Based Uncertainty Quantification in Spatiotemporal Graph Neural Networks for Traffic Forecasting

A review of predictive uncertainty estimation with machine learning

Signal propagation in complex networks

Contact Info

Product

Resources

About