Design of a metadata framework for environmental models with an example hydrologic application in HydroShare

Morsy, Mohamed M.; Goodall, Jonathan L.; Castronova, Anthony M.; Dash, Pabitra; Merwade, Venkatesh; Sadler, Jeffrey M.; Rajib, Adnan; Horsburgh, Jeffery S.; Tarboton, David G.

doi:10.1016/j.envsoft.2017.02.028

Cited by 51 publications

(24 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, the Consortium of Universities for the Advancement of Hydrologic Sciences, Inc. (CUAHSI) HydroShare system is in development and could serve as an alternative or secondary end point for publishing results with more discipline-specific metadata [Horsburgh et al, 2015;Morsy et al, 2014;Tarboton et al, 2014], as could systems such as FigShare or Zenodo. We anticipate a growing number of such repositories and for federation between them (e.g., SEAD is already a member node in DataOne [Michener et al, 2012], advertising our WSO publications through DataONE's catalog).…”

Section: Reproducibilitymentioning

confidence: 99%

Server‐side workflow execution using data grid technology for reproducible analyses of data‐intensive hydrologic systems

Essawy

Goodall

et al. 2016

Earth and Space Science

Self Cite

View full text Add to dashboard Cite

Many geoscience disciplines utilize complex computational models for advancing understanding and sustainable management of Earth systems. Executing such models and their associated data preprocessing and postprocessing routines can be challenging for a number of reasons including (1) accessing and preprocessing the large volume and variety of data required by the model, (2) postprocessing large data collections generated by the model, and (3) orchestrating data processing tools, each with unique software dependencies, into workflows that can be easily reproduced and reused. To address these challenges, the work reported in this paper leverages the Workflow Structured Object functionality of the Integrated Rule-Oriented Data System and demonstrates how it can be used to access distributed data, encapsulate hydrologic data processing as workflows, and federate with other community-driven cyberinfrastructure systems. The approach is demonstrated for a study investigating the impact of drought on populations in the Carolinas region of the United States. The analysis leverages computational modeling along with data from the Terra Populus project and data management and publication services provided by the Sustainable Environment-Actionable Data project. The work is part of a larger effort under the DataNet Federation Consortium project that aims to demonstrate data and computational interoperability across cyberinfrastructure developed independently by scientific communities.

show abstract

Section: Reproducibilitymentioning

confidence: 99%

Server‐side workflow execution using data grid technology for reproducible analyses of data‐intensive hydrologic systems

Essawy

Goodall

et al. 2016

Earth and Space Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…To publish a reproducible version of this research, we used the HydroShare (www.hydroshare.org) cyberinfrastructure platform, designed for reproducing, reusing, and sharing models (Tarboton et al, 2014;Horsburgh et al, 2016;Morsy et al, 2017). Steps that supported reproducibility included using the HydroShare sharing settings with a workflow that started with Private while data and models were developed, Discoverable while research was being shared with colleagues for review, and Public, once our results were accepted for publication.…”

Section: Reproducibilitymentioning

confidence: 99%

A hydroclimatological approach to predicting regional landslide probability using Landlab

Strauch,

Istanbulluoglu,

Nudurupati

et al. 2018

HydroShare Resources

View full text Add to dashboard Cite

Abstract. We develop a hydroclimatological approach to the modeling of regional shallow landslide initiation that integrates spatial and temporal dimensions of parameter uncertainty to estimate an annual probability of landslide initiation based on Monte Carlo simulations. The physically based model couples the infinite-slope stability model with a steady-state subsurface flow representation and operates in a digital elevation model. Spatially distributed gridded data for soil properties and vegetation classification are used for parameter estimation of probability distributions that characterize model input uncertainty. Hydrologic forcing to the model is through annual maximum daily recharge to subsurface flow obtained from a macroscale hydrologic model. We demonstrate the model in a steep mountainous region in northern Washington, USA, over 2700 km 2 . The influence of soil depth on the probability of landslide initiation is investigated through comparisons among model output produced using three different soil depth scenarios reflecting the uncertainty of soil depth and its potential long-term variability. We found elevation-dependent patterns in probability of landslide initiation that showed the stabilizing effects of forests at low elevations, an increased landslide probability with forest decline at midelevations (1400 to 2400 m), and soil limitation and steep topographic controls at high alpine elevations and in post-glacial landscapes. These dominant controls manifest themselves in a bimodal distribution of spatial annual landslide probability. Model testing with limited observations revealed similarly moderate model confidence for the three hazard maps, suggesting suitable use as relative hazard products. The model is available as a component in Landlab, an open-source, Python-based landscape earth systems modeling environment, and is designed to be easily reproduced utilizing HydroShare cyberinfrastructure.

show abstract

“…user can specify which model program is meant to execute the input files contained within the model instance resource (Morsy et al, 2017). Additional HydroShare resource types used in this study include the composite resource type, which allows uploading metadata files at both file and resource level, the collections resource type, which stores any number of resources as a single, aggregate resource, and the web-app resource type, which was described earlier.…”

Section: Hydrosharementioning

confidence: 99%

“…HydroShare divides computational models into two separate but linked resource types: a) the model program and b) the model instance (Morsy et al, 2017). The model program is the software for executing the model and the model instance is the input files required for executing the model and, optionally, the output files after a model instance has been executed by a model program Morsy et al, 2014b).…”

Section: Hydrosharementioning

confidence: 99%

“…The "generic" resource type supports the Dublin Core metadata standard and more specific resource types expand on this metadata for well-defined resource types. For example, "Operating System" is one of the extended metadata terms for the 'Model Program' resource type, which is used for sharing a computational model program in HydroShare (Morsy et al, 2017).…”

Section: Hydrosharementioning

confidence: 99%

See 1 more Smart Citation

Advancing Cyberinfrastructure for Reproducible Hydrologic Modeling

Essawy

View full text Add to dashboard Cite

Scientists have created a significant and growing collection of software tools for data manipulation, analysis, and simulation. This software includes not only computational models, but also a large collection of data pre-and post-processing tools used to support computational modeling and data analysis. This large and diverse set of scientific modeling software presents challenges to the hydrologic science community including (1) the difficulty of making scientific workflows reproducible in support of an end-to-end hydrologic modeling use case, especially when large dataset transfers are required for data processing pipelines, (2) providing adequate and accessible component-level metadata for legacy hydrologic software, and (3) ensuring scientific reproducibility when using legacy hydrologic processing software, which often has complex software dependencies. This research uses the Variable Infiltration Capacity (VIC) and MODFLOW hydrologic models as use cases in examining these challenges.This research addresses these challenges by conducting three studies. The first study explores approaches for leveraging data grid technology in hydrologic modeling to support reproducible workflows using large datasets. Its primary contribution is a general methodology for analyzing large, distributed data collections. This is accomplished by moving processing resources to the large datasets in contrast to the typical approach of moving the datasets to the processing resources. Data grid technology is used to automate data transfers and staging, in combination with automated formal publication of generated data assets.The second study advances prior efforts for formalizing model metadata in hydrology by evaluating the OntoSoft Ontology as a means for formally structuring model metadata for lower level scientific software components. The metadata evaluated describes a data pre-processing ii workflow for the VIC hydrologic model. This workflow consists of multiple software components written by different scientists over time. The analysis begins by exploring what metadata hydrologists have already captured in unstructured forms. It then shows how this metadata could be organized into structured, machine-readable metadata using the OntoSoft Ontology.Finally, the third study explores the creation of containers using Docker to more easily execute hydrologic modeling software in a computing environment. By containerizing a model with all of its dependencies, the model is self-contained and portable. This work contributes a methodology for using HydroShare and Geotrust, two new cyberinfrastructure tools under active development, to improve reproducibility in computational hydrology. HydroShare is a web-based system for sharing hydrologic data and model resources.

show abstract

Design of a metadata framework for environmental models with an example hydrologic application in HydroShare

Cited by 51 publications

References 20 publications

Server‐side workflow execution using data grid technology for reproducible analyses of data‐intensive hydrologic systems

Server‐side workflow execution using data grid technology for reproducible analyses of data‐intensive hydrologic systems

A hydroclimatological approach to predicting regional landslide probability using Landlab

Advancing Cyberinfrastructure for Reproducible Hydrologic Modeling

Contact Info

Product

Resources

About