NASA’s Earth Observing Data and Information System – Near-Term Challenges

Abstract

NASA's Earth Observing System Data and Information System (EOSDIS) has been a central component of the NASA Earth observation program since the 1990's. EOSDIS manages data covering a wide range of Earth science disciplines including cryosphere, land cover change, polar processes, field campaigns, ocean surface, digital elevation, atmospheric dynamics and composition, and inter-disciplinary research, and many others. One of the key components of EOSDIS is a set of twelve discipline-based Distributed Active Archive Centers (DAACs) distributed across the United States. Managed by NASA's Earth Science Data and Information System (ESDIS) Project at the Goddard Space Flight Center, these DAACs serve over 4 million users globally. The ESDIS Project provides the infrastructure support for EOSDIS, which includes other components such as common metadata and metrics management systems, specialized network systems, standards management, and centralized support for use of commercial cloud capabilities. Given the long-term requirements, and the rapid pace of information technology and changing expectations of the user community, EOSDIS has evolved continually over the past three decades. However, many challenges remain. Challenges in three key areas are addressed in this paper: managing volume and variety, enabling data discovery and access, and incorporating user feedback and concerns.

How reproducible and reliable is geophysical research?

“…Indeed, many of the perceived barriers, for example challenges in handling large data files, are not unique to geophysics and these concerns have mostly been shown to be relatively straightforward to manage in terms of absolute volume. For example, a study in neurosciences by Poldrack and Gorgolewski (2014) described how the sharing of raw MRI data from 1,000 authors would consist of ~2.7 terabytes, a relative modest volume by current stor-age infrastructure solutions (e.g., Behnke et al, 2019); however, there are major challenges in ensuring that data sets are curated to make them accessible and useful to researchers. Indeed, the common occurrence of big data within nearly all subjects has served to identify that discussing absolute data volume as a barrier in any context is limiting, as computing hardware and software advances at such a rate that any absolute numbers are soon superseded (Oguntimilehin and Ademola, 2014).…”

Enabling High‐Performance Cloud Computing for Earth Science Modeling on Over a Thousand Cores: Application to the GEOS‐Chem Atmospheric Chemistry Model

“…The 100 TB of the Model Intercomparison Project Phase 6 data is now available in the Google Public Dataset (Google, 2019). The NASA Earth Observing System Data and Information System plans to move petabytes of Earth observation data to the AWS cloud storage (Behnke et al, 2019; Lynnes et al, 2017; Ramachandran et al, 2018). The cloud further offers massive computing power to analyze big data.…”

Enabling high-performance cloud computing for Earth science modeling on over a thousand cores: application to the GEOS-Chem atmospheric chemistry model

“…100 TB of the Model Intercomparison Project Phase 6 (CMIP6) data are now available in the Google Public Dataset (Google, 2019). The NASA Earth Observing System Data and Information System (EOSDIS) plans to move PetaBytes (PBs) of Earth observation data to the AWS cloud storage (Behnke et al, 2019;Lynnes et al, 2017;Ramachandran et al, 2018). The cloud further offers massive computing power to analyze big data.…”