A review of machine learning concepts and methods for addressing challenges in probabilistic hydrological post-processing and forecasting

Papacharalampous, Georgia; Tyralis, Hristos

doi:10.3389/frwa.2022.961954

Cited by 21 publications

(12 citation statements)

References 197 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Lastly, instead of aiming at providing accurate mean-value predictions, one could aim at providing accurate median-value predictions coupled with useful uncertainty estimates. This would require working on machine and statistical learning methods, such as those summarized and popularized in the reviews by [75,76].…”

Section: Discussionmentioning

confidence: 99%

Comparison of Tree-Based Ensemble Algorithms for Merging Satellite and Earth-Observed Precipitation Data at the Daily Time Scale

Papacharalampous

Doulamis

2023

Hydrology

Self Cite

View full text Add to dashboard Cite

Merging satellite products and ground-based measurements is often required for obtaining precipitation datasets that simultaneously cover large regions with high density and are more accurate than pure satellite precipitation products. Machine and statistical learning regression algorithms are regularly utilized in this endeavor. At the same time, tree-based ensemble algorithms are adopted in various fields for solving regression problems with high accuracy and low computational costs. Still, information on which tree-based ensemble algorithm to select for correcting satellite precipitation products for the contiguous United States (US) at the daily time scale is missing from the literature. In this study, we worked towards filling this methodological gap by conducting an extensive comparison between three algorithms of the category of interest, specifically between random forests, gradient boosting machines (gbm) and extreme gradient boosting (XGBoost). We used daily data from the PERSIANN (Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks) and the IMERG (Integrated Multi-satellitE Retrievals for GPM) gridded datasets. We also used earth-observed precipitation data from the Global Historical Climatology Network daily (GHCNd) database. The experiments referred to the entire contiguous US and additionally included the application of the linear regression algorithm for benchmarking purposes. The results suggest that XGBoost is the best-performing tree-based ensemble algorithm among those compared. Indeed, the mean relative improvements that it provided with respect to linear regression (for the case that the latter algorithm was run with the same predictors as XGBoost) are equal to 52.66%, 56.26% and 64.55% (for three different predictor sets), while the respective values are 37.57%, 53.99% and 54.39% for random forests, and 34.72%, 47.99% and 62.61% for gbm. Lastly, the results suggest that IMERG is more useful than PERSIANN in the context investigated.

show abstract

Section: Discussionmentioning

confidence: 99%

Comparison of Tree-Based Ensemble Algorithms for Merging Satellite and Earth-Observed Precipitation Data at the Daily Time Scale

Papacharalampous

Doulamis

2023

Hydrology

Self Cite

View full text Add to dashboard Cite

show abstract

“…Other topics: Methods for probabilistic predictions in geology are surveyed by Albarello and D'Amico (2015). Probabilistic forecasting with machine learning in hydrology is reviewed by Papacharalampous and Tyralis (2022). Metrics for probabilistic predictions are surveyed by Huang and Zhao (2022) in hydroclimatology.…”

Section: Environmental and Earth And Planetary Sciencesmentioning

confidence: 99%

A review of predictive uncertainty estimation with machine learning

Tyralis,

Papacharalampous

2024

Artif Intell Rev

Self Cite

View full text Add to dashboard Cite

Predictions and forecasts of machine learning models should take the form of probability distributions, aiming to increase the quantity of information communicated to end users. Although applications of probabilistic prediction and forecasting with machine learning models in academia and industry are becoming more frequent, related concepts and methods have not been formalized and structured under a holistic view of the entire field. Here, we review the topic of predictive uncertainty estimation with machine learning algorithms, as well as the related metrics (consistent scoring functions and proper scoring rules) for assessing probabilistic predictions. The review covers a time period spanning from the introduction of early statistical (linear regression and time series models, based on Bayesian statistics or quantile regression) to recent machine learning algorithms (including generalized additive models for location, scale and shape, random forests, boosting and deep learning algorithms) that are more flexible by nature. The review of the progress in the field, expedites our understanding on how to develop new algorithms tailored to users’ needs, since the latest advancements are based on some fundamental concepts applied to more complex algorithms. We conclude by classifying the material and discussing challenges that are becoming a hot topic of research.

show abstract

“…Data-driven approaches have gained significant traction in hydrological modeling, prompting numerous review papers [27,28]. These reviews delve into various aspects of DDM in hydrology, including Artificial Neural Network (ANN) applications [17], general DDM techniques [8], AI applications in streamflow modeling [10], ensemble machine learning-based hydrological modeling [29], probabilistic modeling and post-processing [30], and hybrid deep learning-based streamflow forecasting [31].…”

Section: Rational and Contributionmentioning

confidence: 99%

Enhancing Streamflow Prediction Physically Consistently Using Process-Based Modeling and Domain Knowledge: A Review

Yifru,

Lim,

Lee

2024

Sustainability

View full text Add to dashboard Cite

Streamflow prediction (SFP) constitutes a fundamental basis for reliable drought and flood forecasting, optimal reservoir management, and equitable water allocation. Despite significant advancements in the field, accurately predicting extreme events continues to be a persistent challenge due to complex surface and subsurface watershed processes. Therefore, in addition to the fundamental framework, numerous techniques have been used to enhance prediction accuracy and physical consistency. This work provides a well-organized review of more than two decades of efforts to enhance SFP in a physically consistent way using process modeling and flow domain knowledge. This review covers hydrograph analysis, baseflow separation, and process-based modeling (PBM) approaches. This paper provides an in-depth analysis of each technique and a discussion of their applications. Additionally, the existing techniques are categorized, revealing research gaps and promising avenues for future research. Overall, this review paper offers valuable insights into the current state of enhanced SFP within a physically consistent, domain knowledge-informed data-driven modeling framework.

show abstract

A review of machine learning concepts and methods for addressing challenges in probabilistic hydrological post-processing and forecasting

Cited by 21 publications

References 197 publications

Comparison of Tree-Based Ensemble Algorithms for Merging Satellite and Earth-Observed Precipitation Data at the Daily Time Scale

Comparison of Tree-Based Ensemble Algorithms for Merging Satellite and Earth-Observed Precipitation Data at the Daily Time Scale

A review of predictive uncertainty estimation with machine learning

Enhancing Streamflow Prediction Physically Consistently Using Process-Based Modeling and Domain Knowledge: A Review

Contact Info

Product

Resources

About