Caravan - A global community dataset for large-sample hydrology

Kratzert, Frederik; Nearing, Grey; Addor, Nans; Erickson, Tyler; Gauch, Martin; Gilon, Oren; Gudmundsson, Lukas; Hassidim, Avinatan; Klotz, Daniel; Nevo, Sella; Shalev, Guy; Matias, Yossi

doi:10.1038/s41597-023-01975-w

Cited by 78 publications

(27 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The source code used in this research is freely available publicly at: https://github.com/stsfk/indexing_catchment_ model, where a user's manual is also provided to demonstrate the applications of the model instance retrieval method in model calibration. This study uses two freely available datasets: Catchment Attributes and Meteorology for Large-sample Studies (CAMELS; Newman et al (2014); Newman et al (2015); Addor et al (2017)) and Caravan V0.5 (Kratzert et al, 2023). The datasets are freely available at: https://dx.doi.org/10.5065/D6MW2F4D and https://doi.org/10.5281/zenodo.7387919.…”

Section: 1029/2022wr033684mentioning

confidence: 99%

“…The CAMELS (Catchment Attributes and Meteorology for Large-sample Studies; Newman et al (2014); Newman et al (2015); Addor et al (2017)) and Caravan (Kratzert et al, 2023) data sets were used in this study. The CAMELS and Caravan data sets contain daily climate forcing and streamflow time series for 671 US and 6,830 global catchments, respectively.…”

Section: Data and Model Classesmentioning

confidence: 99%

“…(2015); Addor et al. (2017)) and Caravan V0.5 (Kratzert et al., 2023). The datasets are freely available at: https://dx.doi.org/10.5065/D6MW2F4D and https://doi.org/10.5281/zenodo.7387919.…”

Section: Data Availability Statementmentioning

confidence: 99%

See 2 more Smart Citations

Profiling and Pairing Catchments and Hydrological Models With Latent Factor Model

Yang

Chui

2023

Water Resources Research

View full text Add to dashboard Cite

Hydrological models are simplified representations of catchments. Finding an appropriate model instance (i.e., a combination of structure and parameter values) for a given catchment is a fundamental problem in hydrological modeling. This study develops a latent factor‐based machine learning method to learn to predict the pairwise association between the behavioral characteristics under climate forcing of a catchment and a hydrological model instance. The underlying assumption is that the behaviors of catchments and model instances can be sufficiently well characterized by sets of latent factors that allow accurate prediction of the associations between observed and modeled behaviors. In this study, the proposed method was applied to learn the latent factor vectors (which are numerical values) of 2,649 catchments worldwide and over 1,000,000 model instances from their pairwise associations (i.e., similarities between observed and simulated catchment behaviors). The model instances were created using 36 conceptual model structures or modeling software, and the their parameter values were randomly sampled. The results show that the learned latent factor vectors are able to accurately characterize the associations between catchments and hydrological model instances. The latent factor vector of a “new” catchment (not seen by the training procedures) can be estimated by running a small number of randomly sampled model instances, from which well‐fitting model instances created using different conceptual models can be recommended. The recommended instances can be of comparable quality to those derived using a calibration algorithm, and the method is useful for catchments outside the regions from which the latent factor vectors of the model instances were derived.

show abstract

Section: 1029/2022wr033684mentioning

confidence: 99%

Section: Data and Model Classesmentioning

confidence: 99%

See 1 more Smart Citation

Profiling and Pairing Catchments and Hydrological Models With Latent Factor Model

Yang

Chui

2023

Water Resources Research

View full text Add to dashboard Cite

show abstract

“…From earlier uses in small numbers of watersheds, signatures are now frequently calculated in large samples of watersheds spanning national to global scales. This expansion is part of a wider move towards large sample studies that develop datasets and draw conclusions across scales, hydro climates and ecosystems (Addor et al, 2020; Kratzert et al, 2023). Studies that calculate signature values over 100s of gauged watersheds include evaluating national models (Almagro et al, 2021; Coxon et al, 2019; Donnelly et al, 2016; Massmann, 2020; McMillan et al, 2016), selecting model structures (David et al, 2022), interpreting machine learning models (Botterill & McMillan, 2022; Kratzert et al, 2019), predicting signatures from watershed attributes (Addor et al, 2018; Beck et al, 2015; Grantham et al, 2022; Janssen & Ameli, 2021) and classifying watersheds (Kuentz et al, 2017).…”

Section: Introductionmentioning

confidence: 99%

When good signatures go bad: Applying hydrologic signatures in large sample studies

McMillan,

Coxon,

Araki

et al. 2023

Hydrological Processes

View full text Add to dashboard Cite

Hydrologic signatures are quantitative metrics that describe streamflow statistics and dynamics. Signatures have many applications, including assessing habitat suitability and hydrologic alteration, calibrating and evaluating hydrologic models, defining similarity between watersheds and investigating watershed processes. Increasingly, signatures are being used in large sample studies to guide flow management and modelling at continental scales. Using signatures in studies involving 1000s of watersheds brings new challenges as it becomes impractical to examine signature parameters and behaviour in each watershed. For example, we might wish to check that signatures describing flood event characteristics have correctly identified event periods, that signature values have not been biassed by data errors, or that human and natural influences on signature values have been correctly interpreted. In this commentary, we draw from our collective experience to present case studies where naïve application of signatures fails to correctly identify streamflow dynamics. These include unusual precipitation or flow regimes, data quality issues, and signature use in human‐influenced watersheds. We conclude by providing guidance and recommendations on applying signatures in large sample studies.

show abstract

“…(2023) 10:61 | https://doi.org/10.1038/s41597-023-01975-w www.nature.com/scientificdata www.nature.com/scientificdata/…”

mentioning

confidence: 99%

Caravan - A global community dataset for large-sample hydrology

Kratzert¹,

Nearing

Addor

et al. 2023

Preprint

View full text Add to dashboard Cite

High-quality datasets are essential to support hydrological science and modeling. Several datasets exist for specific countries or regions (e.g. the various CAMELS datasets). However, these datasets lack standardization, which makes global studies difficult. Additionally, creating large-sample datasets is a time and resource consuming task, often preventing the release of data that would otherwise be open. Caravan (as in &#8220;a series of camels&#8221;) is an initiative that tries to solve both of these problems by creating an open data processing environment in the cloud for the community to use. Caravan is a globally consistent and open dataset Caravan leverages globally available data sources that are published under an open license to derive meteorological forcings and attributes for any catchment. We use ERA5-Land for meteorological forcings and hydrological reference states (SWE and four levels of soil moisture) and HydroATLAS for the catchment attributes. Currently, Caravan consists of 6830 gauges with daily streamflow data (median record length ~30 years), 9 meteorological variables (from 1981 - 2020) in different daily aggregations, 4 hydrological reference states, and a total of 221 catchment attributes. Caravan is derived entirely in the cloud All meteorological time series (and hydrological reference states) from ERA5-Land are processed on Google Earth Engine, which removes the burden of downloading and processing large amounts of raw gridded data. Similarly, all catchment attributes are computed on Earth Engine. The code used to derive Caravan is publicly available (https://github.com/kratzert/Caravan/) . Once you have streamflow records and the corresponding catchment polygons, deriving all other data (forcing data and attributes) is a matter of a few hours of actual work. Depending on the number of catchments, their size and spatial distribution, that are being processed at once on Earth Engine , it might take a day or two for Earth Engine to extract meteorological data and catchment attributes.&#160; Most importantly: Caravan is a community project Even though the existing data in Caravan has good coverage over most climate zones, the spatial coverage is still patchy. Here is where we see Caravan as a community effort. Given the provided code, everybody with access to streamflow data and the authorisation to redistribute it can create a Caravan extension with minimal effort and share the extension with the community, thus contributing to a dynamically growing dataset. A full step-by-step tutorial is available at https://github.com/kratzert/Caravan/wiki. We envision that, with many people participating, this will result in a truly global and spatially consistent, large-sample hydrology dataset. A first Caravan extension was already published by Julian Koch (https://zenodo.org/record/7396466), which increased the number of gauges to 7138, by adding 308 gauges in Denmark.

show abstract

Caravan - A global community dataset for large-sample hydrology

Cited by 78 publications

References 56 publications

Profiling and Pairing Catchments and Hydrological Models With Latent Factor Model

Profiling and Pairing Catchments and Hydrological Models With Latent Factor Model

When good signatures go bad: Applying hydrologic signatures in large sample studies

Caravan - A global community dataset for large-sample hydrology

Contact Info

Product

Resources

About