A scientific data processing framework for time series NetCDF data

Gaustad, Krista; Shippert, Timothy; Ermold, Brian; Beus, Sherman; Daily, Jeff; Borsholm, Atle; Fox, Kevin M.

doi:10.1016/j.envsoft.2014.06.005

Cited by 9 publications

(2 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They also provide graphical interfaces that facilitate visualization of the workflow and minimize the amount of programming that is necessary to develop the workflow. However, sophisticated scientific workflow systems include a large number of features that are not necessary for every task, and therefore may not always be the most efficient solution for implementing more straightforward scientific data processing workflows (Gaustad et al, 2014). …”

Section: Discussionmentioning

confidence: 99%

Software to facilitate remote sensing data access for disease early warning systems

Liu

Snell-Feikema

et al. 2015

Environmental Modelling & Software

View full text Add to dashboard Cite

Satellite remote sensing produces an abundance of environmental data that can be used in the study of human health. To support the development of early warning systems for mosquito-borne diseases, we developed an open-source, client based software application to enable the Epidemiological Applications of Spatial Technologies (EASTWeb). Two major design decisions were full automation of the discovery, retrieval and processing of remote sensing data from multiple sources, and making the system easily modifiable in response to changes in data availability and user needs. Key innovations that helped to achieve these goals were the implementation of a software framework for data downloading and the design of a scheduler that tracks the complex dependencies among multiple data processing tasks and makes the system resilient to external errors. EASTWeb has been successfully applied to support forecasting of West Nile virus outbreaks in the United States and malaria epidemics in the Ethiopian highlands.

show abstract

Section: Discussionmentioning

confidence: 99%

Software to facilitate remote sensing data access for disease early warning systems

Liu

Snell-Feikema

et al. 2015

Environmental Modelling & Software

View full text Add to dashboard Cite

show abstract

“…This software was released in 2008, and many research studies have been developed by using this tool since then. NCO has different types of commands for reading, writing, interpolating and averaging [7] .…”

Section: Methods Detailsmentioning

confidence: 99%

Processing methodology of global anthropogenic emissions for air quality modeling

Pino-Cortés

2021

MethodsX

View full text Add to dashboard Cite

The Global Emissions Initiative (GEIA) stores and offers global datasets of emission inventories developed in the last 30 years. One of the most recently updated global datasets covering anthropogenic source emissions is the Copernicus Atmosphere Monitoring Service (CAMS). This study applied NetCDF Command Operator (NCO) software to preprocess the anthropogenic sources included in the CAMS datasets and converted those files as an input in the Sparse Matrix Operator Kerner Emissions (SMOKE) model for future air quality modeling. As a result, six steps were applied to obtain the required file format. The case of the central coast in Chile was analyzed to compare the global database and official reports for the on-road transport sector. As a result, some differences were shown in the most populated locations of the domain of analysis. The rest of the zones registered similar values. The methodology exposed in this report could be applied in any other region of the planet for air quality modeling studies. The development of global datasets such as CAMS is useful for hemispheric analysis and could bring an estimation on the mesoscale. It represents an opportunity for those locations without official reports of non-updated data. This study applied NCO commands available for the preprocessing of the CAMS dataset files. The emissions and temporal profile registered in CAMS datasets must be compared to official reports of transport sectors. The development of global datasets such as CAMS is useful for hemispheric analysis and could bring an estimation on the mesoscale.

show abstract

An architecture for consolidating multidimensional time-series data onto a common coordinate grid

Shippert¹,

Gaustad²

2016

Earth Sci Inform

Self Cite

View full text Add to dashboard Cite

Consolidating measurement data for use by data models or in inter-comparison studies frequently requires transforming the data onto a common grid. Standard methods for interpolating multidimensional data are often not appropriate for data with non-homogenous dimensionality, and are hard to implement in a consistent manner for different datastreams. These challenges are increased when dealing with the automated procedures necessary for use with continuous, operational datastreams. In this paper we introduce a method of applying a series of one-dimensional transformations to merge data onto a common grid, examine the challenges of ensuring consistent application of data consolidation methods, present a framework for addressing those challenges, and describe the implementation of such a framework for the Atmospheric Radiation Measurement (ARM) program.

show abstract

A scientific data processing framework for time series NetCDF data

Cited by 9 publications

References 20 publications

Software to facilitate remote sensing data access for disease early warning systems

Software to facilitate remote sensing data access for disease early warning systems

Processing methodology of global anthropogenic emissions for air quality modeling

An architecture for consolidating multidimensional time-series data onto a common coordinate grid

Contact Info

Product

Resources

About