Community‐wide validation of geospace model ground magnetic field perturbation predictions to support model transition to operations

Abstract

[1] In this paper we continue the community-wide rigorous modern space weather model validation efforts carried out within GEM, CEDAR and SHINE programs. In this particular effort, in coordination among the Community Coordinated Modeling Center (CCMC), NOAA Space Weather Prediction Center (SWPC), modelers, and science community, we focus on studying the models' capability to reproduce observed ground magnetic field fluctuations, which are closely related to geomagnetically induced current phenomenon. One of the primary motivations of the work is to support NOAA SWPC in their selection of the next numerical model that will be transitioned into operations. Six geomagnetic events and 12 geomagnetic observatories were selected for validation. While modeled and observed magnetic field time series are available for all 12 stations, the primary metrics analysis is based on six stations that were selected to represent the high-latitude and mid-latitude locations. Events-based analysis and the corresponding contingency tables were built for each event and each station. The elements in the contingency table were then used to calculate Probability of Detection (POD), Probability of False Detection (POFD) and Heidke Skill Score (HSS) for rigorous quantification of the models' performance. In this paper the summary results of the metrics analyses are reported in terms of POD, POFD and HSS. More detailed analyses can be carried out using the event by event contingency tables provided as an online appendix. An online interface built at CCMC and described in the supporting information is also available for more detailed time series analyses.

Climate, weather, space weather: model development in an operational context

“…The approach is interesting as it allows to identify systematic deficiencies across models. In practical terms this also means assistance of community wide model validation efforts (e.g., Pulkkinen et al, 2013;Rastätter et al, 2014;Glocer et al, 2016;Welling et al, 2017). SWP here is a trend setter.…”

Space Weather Quarterly Volume 14, Issue 4, 2017

“…Finally, ground magnetometer observations are used to validate several global models of geospace dynamics for both research and forecast purposes (e.g., Pulkkinen et al, 2013;Shi et al, 2008;Weygand et al, 2011;Yu & Ridley, 2008). The increasing demand for such comparisons again requires the prompt dissemination of all ground magnetometer data, as well as additional higher-level data products.…”

What Controls the Structure and Dynamics of Earth’s Magnetosphere?

“…This single example shows (but does not prove in general) that a more sophisticated model can provide better agreement with measurements. In a recent study, requested by the U.S. national Space Weather Prediction Center (SWPC), Pulkkinen et al (2013) compared several empirical and physics based magnetosphere models for several geomagnetic events and found that the SWMF using the coupled BATS-R-US, RCM and RIM models consistently outperformed all the empirical models as well as the global MHD models that were not coupled with an inner magnetosphere model, and in fact the SWMF has been selected to be transitioned to operations at NOAA's Space Weather Prediction Center (SWPC).…”