Revealing the Statistics of Extreme Events Hidden in Short Weather Forecast Data

Finkel, Justin; Gerber, Edwin P.; Abbot, Dorian S.; Weare, Jonathan

doi:10.1029/2023av000881

Cited by 11 publications

(4 citation statements)

References 105 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The "diffusive" model fixes α = 0.50, while the "subdiffusive" model adapts α to minimize the cost. E et al 2004;Plotkin et al 2019;Woillez & Bouchet 2020;Schorlepp et al 2023) rare event schemes, aided by machine-learning predictor functions (Ma & Dinner 2005;Chattopadhyay et al 2020;Finkel et al 2021;Miloshevich et al 2023;Finkel et al 2023).…”

Section: Discussionmentioning

confidence: 99%

Mercury’s Chaotic Secular Evolution as a Subdiffusive Process

Abbot,

Webber,

Hernandez

et al. 2024

ApJ

View full text Add to dashboard Cite

Mercury’s orbit can destabilize, generally resulting in a collision with either Venus or the Sun. Chaotic evolution can cause g 1 to decrease to the approximately constant value of g 5 and create a resonance. Previous work has approximated the variation in g 1 as stochastic diffusion, which leads to a phenomological model that can reproduce the Mercury instability statistics of secular and N-body models on timescales longer than 10 Gyr. Here we show that the diffusive model significantly underpredicts the Mercury instability probability on timescales less than 5 Gyr, the remaining lifespan of the solar system. This is because g 1 exhibits larger variations on short timescales than the diffusive model would suggest. To better model the variations on short timescales, we build a new subdiffusive phenomological model for g 1. Subdiffusion is similar to diffusion but exhibits larger displacements on short timescales and smaller displacements on long timescales. We choose model parameters based on the behavior of the g 1 trajectories in the N-body simulations, leading to a tuned model that can reproduce Mercury instability statistics from 1–40 Gyr. This work motivates fundamental questions in solar system dynamics: why does subdiffusion better approximate the variation in g 1 than standard diffusion? Why is there an upper bound on g 1, but not a lower bound that would prevent it from reaching g 5?

show abstract

Section: Discussionmentioning

confidence: 99%

Mercury’s Chaotic Secular Evolution as a Subdiffusive Process

Abbot,

Webber,

Hernandez

et al. 2024

ApJ

View full text Add to dashboard Cite

show abstract

“…Wu & Chang, 2003). The problem becomes particularly significant when one aims to learn rare/extreme events (Baldi et al, 2014;Chattopadhyay, Nabizadeh, & Hassanzadeh, 2020;Finkel et al, 2023;Liu et al, 2016;Maalouf & Siddiqi, 2014;Maalouf & Trafalis, 2011;Miloshevich et al, 2023;O'Gorman & Dwyer, 2018;Qi & Majda, 2020;Shamekh et al, 2023). For example, suppose that we aim to learn the binary classification of the 99 percentile of temperature anomalies using an NN.…”

Section: Introductionmentioning

confidence: 99%

Data Imbalance, Uncertainty Quantification, and Transfer Learning in Data‐Driven Parameterizations: Lessons From the Emulation of Gravity Wave Momentum Transport in WACCM

Sun,

Pahlavan,

Chattopadhyay

et al. 2024

J Adv Model Earth Syst

Self Cite

View full text Add to dashboard Cite

Neural networks (NNs) are increasingly used for data‐driven subgrid‐scale parameterizations in weather and climate models. While NNs are powerful tools for learning complex non‐linear relationships from data, there are several challenges in using them for parameterizations. Three of these challenges are (a) data imbalance related to learning rare, often large‐amplitude, samples; (b) uncertainty quantification (UQ) of the predictions to provide an accuracy indicator; and (c) generalization to other climates, for example, those with different radiative forcings. Here, we examine the performance of methods for addressing these challenges using NN‐based emulators of the Whole Atmosphere Community Climate Model (WACCM) physics‐based gravity wave (GW) parameterizations as a test case. WACCM has complex, state‐of‐the‐art parameterizations for orography‐, convection‐, and front‐driven GWs. Convection‐ and orography‐driven GWs have significant data imbalance due to the absence of convection or orography in most grid points. We address data imbalance using resampling and/or weighted loss functions, enabling the successful emulation of parameterizations for all three sources. We demonstrate that three UQ methods (Bayesian NNs, variational auto‐encoders, and dropouts) provide ensemble spreads that correspond to accuracy during testing, offering criteria for identifying when an NN gives inaccurate predictions. Finally, we show that the accuracy of these NNs decreases for a warmer climate (4 × CO2). However, their performance is significantly improved by applying transfer learning, for example, re‐training only one layer using ∼1% new data from the warmer climate. The findings of this study offer insights for developing reliable and generalizable data‐driven parameterizations for various processes, including (but not limited to) GWs.

show abstract

“…Although it is not routinely used in space science, EVA is used more commonly in other fields to predict the return periods of for example, earthquakes or extreme weather (e.g., Finkel et al., 2023). EVA has also been used by some authors to predict the return period or probability of extreme Space Weather events.…”

Section: Introductionmentioning

confidence: 99%

Extreme Value Analysis of Ground Magnetometer Observations at Valentia Observatory, Ireland

et al. 2023

View full text Add to dashboard Cite

Ground based magnetometers measure deflections in the magnetic field as a result of changes in overhead currents, which can be used to infer variations in geomagnetically induced currents (GICs, Blake et al., 2016, geoelectric fields (Campanyà et al., 2019;Malone-Leigh et al., 2023), and the overhead currents themselves (e.g., auroral electrojet indices, Davis & Sugiura, 1966). Ground based magnetometers have been used for over a century to characterize space weather effects. As the amount of data recorded at stations builds up over time, a wealth of information can be unraveled from these expanding datasets using new and/or more computationally intensive data analysis techniques (such as machine learning approaches (e.g., James et al., 2021;Marangio et al., 2020), numerical techniques (e.g., Elvidge, 2020) and visualizations (e.g., Chapman et al., 2020)).Space weather impacts a breadth of human technology, and some technology is more susceptible than others. For example, recently 38 starlink satellites were lost to a minor-moderate size geomagnetic storm (Fang et al., 2022), while infrastructure like power grids are more robust to all but the most extreme events (Bolduc, 2002). As we move to an increasingly technologically-reliant society, these space weather risks become of greater importance, and indeed space weather is listed on national risk registers in, for example, the United Kingdom (HM Government Cabinet Office, 2020). Therefore the motivation for understanding, and indeed predicting the most extreme space weather events is of the utmost importance.

show abstract

Revealing the Statistics of Extreme Events Hidden in Short Weather Forecast Data

Cited by 11 publications

References 105 publications

Mercury’s Chaotic Secular Evolution as a Subdiffusive Process

Mercury’s Chaotic Secular Evolution as a Subdiffusive Process

Data Imbalance, Uncertainty Quantification, and Transfer Learning in Data‐Driven Parameterizations: Lessons From the Emulation of Gravity Wave Momentum Transport in WACCM

Extreme Value Analysis of Ground Magnetometer Observations at Valentia Observatory, Ireland

Contact Info

Product

Resources

About