Enabling Immediate Access to Earth Science Models through Cloud Computing: Application to the GEOS-Chem Model

Zhuang, Jiawei; Jacob, Daniel; Gaya, Judith Flo; Yantosca, Robert M.; Lundgren, Elizabeth W.; Sulprizio, Melissa P.; Eastham, Sebastian D.

doi:10.1175/bams-d-18-0243.1

Cited by 19 publications

(29 citation statements)

References 67 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Basic benefits of cloud computing for Earth science modeling are reviewed by Zhuang et al (2019), including (1) immediate access to complex models for novice and occasional users, (2) sharing of model configurations and data sets for collaborative work, and (3) cost‐effective alternative to the acquisition of a local cluster. Here we review additional benefits that have fueled increasing recent interest in cloud computing.…”

Section: Benefits Of Cloud Computing For Earth Science Researchmentioning

confidence: 99%

“…For novice users, it is important to first get familiar with the single‐node workflow (Zhuang et al 2019), which is often sufficient for early‐stage model testing and data analysis tasks. Figure 1a shows the single‐node workflow.…”

Section: Hpc Workflow On the Aws Cloudmentioning

confidence: 99%

“…Single‐node and multinode workflows on the AWS cloud. The single‐node workflow (a) is a simplified version of Figure 2 in Zhuang et al (2019), where the Amazon Machine Image (AMI) contains the software environment for the virtual machine (EC2 instance). The multinode workflow (b) adds the autoscaling group for compute nodes.…”

Section: Hpc Workflow On the Aws Cloudmentioning

confidence: 99%

“…Cloud computing platforms can provide scientists immediate access to complex Earth science models and large data sets, thus greatly facilitating scientific research and collaboration (Zhuang et al, 2019). Past work has ported a number of Earth science models to the cloud (Vance et al, 2016), including the Weather Research and Forecast (WRF) Model (Molthan et al, 2015), the Community Earth System Model (Chen et al, 2017), the Model for Prediction Across Scales‐Ocean (Coffrin et al, 2019), and the Goddard Earth Observing System (GEOS)‐Chem chemical transport model (Zhuang et al, 2019). Beyond proof‐of‐concept demonstrations, organizations are also starting to provide operational support for models on cloud.…”

Section: Introductionmentioning

confidence: 99%

“…It is used by a large international community for a wide range of atmospheric chemistry applications. We have previously developed a user‐ready capability for running single‐node GEOS‐Chem simulations on the AWS cloud (Zhuang et al, 2019), including immediate access to 30 terabytes (TB) of GEOS meteorological data and 4 TB of emission data (Keller et al, 2014) through the AWS cloud storage. Here we push this development further to high‐performance, multinode applications.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Enabling High‐Performance Cloud Computing for Earth Science Modeling on Over a Thousand Cores: Application to the GEOS‐Chem Atmospheric Chemistry Model

Zhuang

Jacob

Lin

et al. 2020

J Adv Model Earth Syst

Self Cite

View full text Add to dashboard Cite

Cloud computing platforms can facilitate the use of Earth science models by providing immediate access to fully configured software, massive computing power, and large input data sets. However, slow internode communication performance has previously discouraged the use of cloud platforms for massively parallel simulations. Here we show that recent advances in the network performance on the Amazon Web Services cloud enable efficient model simulations with over a thousand cores. The choices of Message Passing Interface library configuration and internode communication protocol are critical to this success. Application to the Goddard Earth Observing System (GEOS)-Chem global 3-D chemical transport model at 50-km horizontal resolution shows efficient scaling up to at least 1,152 cores, with performance and cost comparable to the National Aeronautics and Space Administration Pleiades supercomputing cluster.Plain Language Summary Earth science model simulations are computationally expensive, typically requiring the use of high-end supercomputing clusters that are managed by universities or national laboratories. Commercial cloud computing offers an alternative. However, past work found that cloud computing platforms were not efficient for large-scale simulations on over 100 CPU cores, because the network communication performance on the cloud was slow compared to local clusters. Here we show that recent advances in the cloud network performance enable efficient model simulations with over a thousand cores, and cloud platforms can now serve as a viable alternative to local clusters for simulations at large scale. Computing on the cloud has extensive advantages, such as providing immediate access to fully configured model code and large data sets for any users, allowing full reproducibility of model simulation results, offering quick access to novel hardware that might not be available on local clusters, and being able to scale to virtually unlimited amounts of compute and storage resources. Those benefits will help advance Earth science modeling research.

show abstract

Section: Benefits Of Cloud Computing For Earth Science Researchmentioning

confidence: 99%

Section: Hpc Workflow On the Aws Cloudmentioning

confidence: 99%