The good, the bad and the outliers: automated detection of errors and outliers from groundwater hydrographs

Peterson, T. J.; Western, Andrew W.; Chen, Xiang

doi:10.1007/s10040-017-1660-7

Cited by 26 publications

(28 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Von Asmuth et al (2002) introduced a new type of TFN models based on the principles of convolution and predefined impulse response functions. This type of model has been applied in a variety of studies including the decomposition of hydrological stresses (von Asmuth and Knotters 2004;von Asmuth et al 2008;Shapoori et al 2015b), the estimation of aquifer parameters (e.g., Obergfell et al 2013;Shapoori et al 2015a), the statistical interpolation of groundwater time series (Peterson et al 2018), the analysis of nation-wide groundwater monitoring networks (Zaadnoordijk et al 2018), and the estimation of recharge (Obergfell et al 2019). The concepts of TFN modeling based on impulse response functions have been incorporated in the commercially available software Menyanthes (von Asmuth et al 2012) and the open source software HydroSight (Peterson and Western 2014).…”

Section: Introductionmentioning

confidence: 99%

Pastas: Open Source Software for the Analysis of Groundwater Time Series

Collenteur¹,

Bakker

Caljé³

et al. 2019

Groundwater

View full text Add to dashboard Cite

Time series analysis is an increasingly popular method to analyze heads measured in an observation well. Common applications include the quantification of the effect of different stresses (rainfall, pumping, etc.), and the detection of trends and outliers. Pastas is a new and open source Python package for the analysis of hydrogeological time series. The objective of Pastas is twofold: to provide a scientific framework to develop and test new methods, and to provide a reliable ready‐to‐use software tool for groundwater practitioners. Transfer function noise modeling is applied using predefined response functions. For example, the head response to rainfall is simulated through the convolution of measured rainfall with a Gamma response function. Pastas models are created and analyzed through scripts, ensuring reproducibility and providing a transparent report of the entire modeling process. A Pastas model can be constructed in seven simple steps: import Pastas, read the time series, create a model, specify the stresses and the types of response functions, estimate the model parameters, visualize output, and analyze the results. These seven steps, including the corresponding Python code, are applied to investigate how rainfall and reference evaporation can explain measured heads in an observation well in Kingstown, Rhode Island, USA. The second example demonstrates the use of scripts to analyze a large number of observation wells in batch to estimate the extent of the drawdown caused by a well field in the Netherlands. Pastas is free and open source software available under the MIT‐license at http://github.com/pastas/pastas.

show abstract

Section: Introductionmentioning

confidence: 99%

Pastas: Open Source Software for the Analysis of Groundwater Time Series

Collenteur¹,

Bakker

Caljé³

et al. 2019

Groundwater

View full text Add to dashboard Cite

show abstract

“…However, with the mentioned large, non-scientific datasets as used in our model case, and when this is done prior to the parameter estimation, it is hard to determine which observations with large residuals represent observational outliers, or model structural deficiencies, or non-optimal parameter values. Statistically sound outlier filtering often requires timeseries of observations (Jeong et al, 2017;Peterson et al, 2018), whereas our observation dataset is comprised of many observation points with only a single or few observations in time, and only few observation points with a whole time series. Furthermore, outliers can also be detected based on spatial patterns (Bárdossy and Kundzewicz, 1990).…”

Section: Discussionmentioning

confidence: 99%

A robust objective function for calibration of groundwater models in light of deficiencies of model structure and observations

Schneider

Henriksen

Stisen

2020

Preprint

View full text Add to dashboard Cite

Abstract. Groundwater models require parameter optimization based on the minimization of objective functions describing, for example, the residual between observed and simulated groundwater head. At larger scales, constraining these models requires large datasets of groundwater head observations, due to the size of the inverse problem. These observations are typically only available from databases comprised of varying quality data from a variety of sources and will be associated with unknown observational uncertainty. At the same time the model structure, especially the hydrogeological description, will inevitably be a simplification of the complex natural system. As a result, calibration of groundwater models often results in parameter compensation for model structural deficiency. This problem can be amplified by the application of common squared error-based performance criteria, which are most sensitive to the largest errors. We assume that the residuals that remain large during the optimization process likely do so because of either model structural error or observation error. Based on this assumption it is desirable to design an objective function that is less sensitive to these large residuals of low probability, and instead favours the majority of observations that can fit the given model structure. We suggest a Continuous Ranked Probability Score (CRPS) based objective function that limits the influence of large residuals in the optimization process as the metric puts more emphasis on the position of the residual along the cumulative distribution function than on the magnitude of the residual. The CRPS-based objective function was applied in two regional scale coupled surface-groundwater models and compared to calibrations using conventional sum of absolute and squared errors. The optimization tests illustrated that the novel CRPS-based objective function successfully limited the dominance of large residuals in the optimization process and consistently reduced overall bias. Furthermore, it highlighted areas in the model where the structural model should be revisited.

show abstract

“…To assess the approach, the following procedure was applied to each bore for hydrograph interpolation. Identify and remove erroneous and outlier water level observations using the automatic procedure from Peterson et al (). Erroneous observations were identified using heuristics, for example, maximum acceptable level change per day and the maximum acceptable duration of periods of constant water level.…”

Section: Methodsmentioning

confidence: 99%

“…1. Identify and remove erroneous and outlier water level observations using the automatic procedure from Peterson et al (2017)…”

Section: Evaluation Of Interpolation Proceduresmentioning

confidence: 99%

Statistical Interpolation of Groundwater Hydrographs

Peterson

Western

2018

Water Resources Research

Self Cite

View full text Add to dashboard Cite

Groundwater observation bores are often monitored irregularly and infrequently. The resulting groundwater hydrographs are consequently less informative for understanding groundwater level trends, seasonality, flow directions, drawdown, and recovery. This paper presents an approach to temporally interpolate a groundwater hydrograph that has an irregular observation frequency to daily time steps. The approach combines nonlinear transfer function noise modeling with temporal kriging of the model residuals to produce an interpolated hydrograph that honors all water level observations input to the modeling and accounts for meteorological forcing between the observations. The reliability of the approach was evaluated using six observation bores having extended periods of daily data and by resampling them to six observation frequencies ranging from weekly to annually. The analysis showed that for weekly to monthly resampled data, >90% of the observed daily variability can be simulated at four of six bores. The performance declined with observation step size, as expected, but even at a biannual time step the error corrected interpolation can explain >70% of the variance at three of six bores. Additionally, an application shows that (1) the probability of a water level depth being exceeded can be estimated from quarterly resampled data and (2) the median annual water level range can be estimated from monthly resampled data. Supplementing less frequent observations with 6 and 12 months of daily data was also examined, with the addition of a 12‐month period significantly improving interpolation results at three of the four analyzed bores. The approach has been incorporated into the HydroSight toolbox http://peterson-tim-j.github.io/HydroSight/.

show abstract

The good, the bad and the outliers: automated detection of errors and outliers from groundwater hydrographs

Cited by 26 publications

References 16 publications

Pastas: Open Source Software for the Analysis of Groundwater Time Series

Pastas: Open Source Software for the Analysis of Groundwater Time Series

A robust objective function for calibration of groundwater models in light of deficiencies of model structure and observations

Statistical Interpolation of Groundwater Hydrographs

Contact Info

Product

Resources

About