Abstract:We present an overview of key aspects of the Atmospheric Radiation Measurement (ARM) Program Climate Research Facility (ACRF) data quality assurance program. Processes described include instrument deployment and calibration; instrument and facility maintenance; data collection and processing infrastructure; data stream inspection and assessment; problem reporting, review and resolution; data archival, display and distribution; data stream reprocessing; engineering and operations management; and the roles of value-added data processing and targeted field campaigns in specifying data quality and characterizing field measurements. The paper also includes a discussion of recent directions in ACRF data quality assurance. A comprehensive, end-to-end data quality assurance program is essential for producing a high-quality data set from measurements made by automated weather and climate networks. The processes developed during the ARM Program offer a possible framework for use by other instrumentation-and geographically-diverse data collection networks and highlight the myriad aspects that go into producing research-quality data.

No abstract

Abstract. Global climate researchers rely upon many forms of sensor data and analytical methods to help profile subtle changes in climate conditions. The U.S. Department of Energy's Atmospheric Radiation Measurement (ARM) program provides researchers with a collection of curated Value Added Products (VAPs) resulting from continuous sensor data streams, data fusion, and modeling. We are leveraging the Open Provenance Model as a foundational construct that serves the needs of both the VAP producers and consumers. We are organizing the provenance in different tiers of granularity to model VAP lineage, causality at the component level within a VAP, and the causality for each time step as samples are being assembled within the VAP. This paper shares our implementation strategy and how the ARM operations staff and the climate research community can greatly benefit from this approach to more effectively assess and quantify VAP provenance.

Global climate researchers rely upon many forms of sensor data and analytical methods to help profile subtle changes in climate conditions. The U.S. Department of Energy Atmospheric Radiation Measurement (ARM) program provides researchers with curated Value Added Products (VAPs) resulting from continuous instrumentation streams, data fusion, and analytical profiling. The ARM operational staff and software development teams (data producers) rely upon a number of techniques to ensure strict quality control (QC) and quality assurance (QA) standards are maintained. Climate researchers (data consumers) are highly interested in obtaining as much provenance evidence as possible to establish data trustworthiness. Currently all the evidence is not easily attainable or identifiable without significant efforts to extract and piece together information from configuration files, log files, codes, or status information on the ARM website. Our objective is to identify a provenance model that serves the needs of both the VAP producers and consumers. This paper shares our initial results -a comprehensive multi-tier provenance model. We describe how both ARM operations staff and the climate research community can greatly benefit from this approach to more effectively assess and quantify the data historical record.

Abstract. The US Department of Energy (DOE) Atmospheric Radiation Measurement Program (ARM) is adopting the use of formalized provenance to support observational data products produced by ARM operations and relied upon by researchers. Because of the diversity of needs in the climate community provenance will need to be conveyed in a domain-oriented context. This paper explores a use case where semantic abstract workflows (SAW) are employed as a means to filter, aggregate, and contextually describe the historical events responsible for the ARM data product the scientist is relying upon.