MOLES3: Implementing an ISO standards driven data catalogue

Abstract

ISO19156 Observations and Measurements (O&M) provides a standardised framework for organising information about the collection of information about the environment. Here we describe the implementation of a specialisation of O&M for environmental data, the Metadata Objects for Linking Environmental Sciences (MOLES3).MOLES3 provides support for organising information about data, and for user navigation around data holdings. The implementation described here, “CEDA-MOLES”, also supports data management functions for the Centre for Environmental Data Archival, CEDA. The previous iteration of MOLES (MOLES2) saw active use over five years, being replaced by CEDA-MOLES in late 2014. During that period important lessons were learnt both about the information needed, as well as how to design and maintain the necessary information systems. In this paper we review the problems encountered in MOLES2; how and why CEDA-MOLES was developed and engineered; the migration of information holdings from MOLES2 to CEDA-MOLES; and, finally, provide an early assessment of MOLES3 (as implemented in CEDA-MOLES) and its limitations.Key drivers for the MOLES3 development included the necessity for improved data provenance, for further structured information to support ISO19115 discovery metadata export (for EU INSPIRE compliance), and to provide appropriate fixed landing pages for Digital Object Identifiers (DOIs) in the presence of evolving datasets. Key lessons learned included the importance of minimising information structure in free text fields, and the necessity to support as much agility in the information infrastructure as possible without compromising on maintainability both by those using the systems internally and externally (e.g. citing in to the information infrastructure), and those responsible for the systems themselves. The migration itself needed to ensure continuity of service and traceability of archived assets.

Developing an Open Data Portal for the ESA Climate Change Initiative

“…The Centre for Environmental Data Analysis (CEDA) provides the necessary archival and curation environment -archival provides reliable data availability including multiple copies of data and curation provides active management of the data, including format and metadata migration (if necessary). Systems have been built up over decades (Pepler and Callaghan, 2015) to deliver reliable data services and complex metadata systems are in place (Parton et al, 2015) which exploit a wide range of information categories from archive metadata, to the necessary browse and discovery metadata (Lawrence et al, 2009). These data systems are supported by data scientists who have expertise in the relevant science areas and can actively curate the metadata content to maximise utility as systems and target communities evolve.…”

“…The development of the metadata systems for supporting the workflow necessary to publish data into the portal and out to users formed a major component of the work. This architecture depends on several key underlying technologies: the ESGF Publisher which indexes key metadata from NetCDF source files into the Solr distributed search system; 9 both the ESGF and the Web Presence utilise faceted search based on the Data Reference Syntax (DRS, Petrie et al, 2020); and the metadata systems depend on the CEDA catalogue system (which uses the Metadata Objects for Linking Environmental Sciences, MOLES, Parton et al, 2015).…”

Data rescue archive weather (DRAW)

“…Next, domain-oriented metadata schemas could be considered (e.g. ISO 19115 for geospatial data (International Organization for Standardization, 2014) the Metadata Objects for Linking Environmental Sciences schema for environmental science data (Parton et al , 2015), etc.). However, those metadata standards are too specialized and narrow to fit heterogeneous variations of data sets.…”