More accuracy with less precision

Lang, Simon; Dawson, Andrew; Diamantakis, Michail; Dueben, Peter; Hatfield, Sam; Leutbecher, Martin; Palmer, Tim; Prates, Fernando; Roberts, C. D.; Sandu, Irina; Wedi, Nils

doi:10.1002/qj.4181

Cited by 23 publications

(12 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Applied to binary numbers in simple chaotic dynamical systems, the information is zero for many of the 32 bits in Float32 22 . This supports the general concept of low-precision climate modeling for calculations and data storage, as, at least in theory, many rounding errors are entirely masked by other uncertainties in the chaotic climate system [23][24][25] .…”

supporting

confidence: 71%

Compressing atmospheric data into its real information content

et al. 2021

View full text Add to dashboard Cite

Hundreds of petabytes are produced annually at weather and climate forecast centers worldwide. Compression is essential to reduce storage and to facilitate data sharing. Current techniques do not distinguish the real from the false information in data, leaving the level of meaningful precision unassessed. Here we define the bitwise real information content from information theory for the Copernicus Atmospheric Monitoring Service (CAMS). Most variables contain fewer than 7 bits of real information per value and are highly compressible due to spatio-temporal correlation. Rounding bits without real information to zero facilitates lossless compression algorithms and encodes the uncertainty within the data itself. All CAMS data are 17× compressed relative to 64-bit floats, while preserving 99% of real information. Combined with four-dimensional compression, factors beyond 60× are achieved. A data compression Turing test is proposed to optimize compressibility while minimizing information loss for the end use of weather and climate forecast data.

show abstract

supporting

confidence: 71%

Compressing atmospheric data into its real information content

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Since their EnKF is implemented as a subroutine within the CGCM, all CDA procedures are conducted rapidly in the memory by avoiding frequent I/O of restart files. Other promising solutions include running the CGCM and its CDA package with reduced precisions [Váňa et al, 2017;Lang et al, 2021], and developing emulators for the CGCM using machine learning and Artificial Intelligence techniques [Pathak et al, 2022;Lam et al, 2022]. Besides computational resource challenges, extending the SCDA approach to more coupled earth system components is also desirable.…”

Section: Summary and Discussionmentioning

confidence: 99%

Towards Strongly-coupled Ensemble Data Assimilation with Additional Improvements from Machine Learning

Kalnay

Sluka

Yoshida

et al. 2023

Preprint

View full text Add to dashboard Cite

Abstract. We assessed different coupled data assimilation strategies with a hierarchy of coupled models, ranging from the simple coupled Lorenz model to the state-of-the-art coupled general circulation model CFSv2. With the coupled Lorenz model, we assessed the analysis accuracy by strongly-coupled Ensemble Kalman Filter (EnKF) and 4D-Variational (4D-Var) methods with varying assimilation window lengths. The analysis accuracy of the strongly-coupled EnKF with a short assimilation window is comparable to that of 4D-Var with a long assimilation window. For 4D-Var, the strongly-coupled approach with the coupled model produces more accurate ocean analysis than the ECCO-like approach using the uncoupled ocean model. Experiments with the coupled quasi-geostrophic model conclude that the strongly-coupled approach outperforms the weakly-coupled and uncoupled approaches for both the full-rank EnKF and 4D-Var, with the strongly-coupled EnKF and 4D-Var showing a similar level of accuracy higher than other coupled data assimilation approaches such as the outer loop coupling. A strongly-coupled EnKF software framework is developed and applied to the intermediate-complexity coupled model SPEEDY-NEMO and the state-of-the-art operational coupled model CFSv2. Experiments assimilating synthetic or real atmospheric observations into the ocean through strongly-coupled EnKF show that the strongly-coupled approach improves the analysis of the atmosphere and upper oceans, but degrades observation fits in the deep ocean, probably due to the unreliable error correlation estimated by a small ensemble. The correlation-cutoff method is developed to reduce the unreliable error correlations between physically irrelevant model states and observations. Experiments with the coupled Lorenz model demonstrate that strongly-coupled EnKF informed by the correlation-cutoff method produces more accurate coupled analyses than the weakly-coupled and plain strongly-coupled EnKF regardless of the ensemble size. To extend the correlation-cutoff method to operational coupled models, a neural network approach is proposed to systematically acquire the observation localization functions for all pairs between the model state and observation types. The following strongly-coupled EnKF experiments with an intermediate-complexity coupled model show promising results with this method.

show abstract

“…The forecast experiments presented here are based on IFS model cycle 47R1 in single precision coupled with a 0.25° ocean model NEMO, that is, a configuration compatible with operational model cycle 47R2 which introduced single precision (Váňa et al ., 2017; Lang et al ., 2021) in IFS on 11 May 2021. Two types of experiment are presented.…”

Section: Weather Forecast Tests and Verificationmentioning

confidence: 99%

A fast converging and concise algorithm for computing the departure points in semi‐Lagrangian weather and climate models

Diamantakis

Váňa

2021

Quart J Royal Meteoro Soc

Self Cite

View full text Add to dashboard Cite

Accurate calculation of departure points is crucial for the overall accuracy of a semi-Lagrangian advection scheme. The iterative method used in the semi-Lagrangian advection scheme of the ECMWF model IFS (Integrated Forecast System) is based on SETTLS (Stable Extrapolation Two-Time-Level Scheme). The scheme converges slowly due to the long timesteps that the spectral semi-implicit, semi-Lagrangian formulation permits: five iterations are needed for accurate determination of the departure points. Spherical coordinates increase the expense and complexity of the relevant code while computations become sensitive to arithmetic precision and require further approximations in the polar high-curvature region. In this article, an accurate alternative, derived by reformulating the current SETTLS scheme on a geocentric Cartesian framework, is proposed. The new scheme simplifies considerably the computations by avoiding the update of the Lagrangian trajectory equations in spherical coordinates, which involve additional expensive metric terms, eliminates approximations, and is advantageous for single precision. Methods to accelerate the convergence of the departure points were also investigated. The most accurate and efficient approach was to start the SETTLS iterations from the departure point of the previous timestep. Testing the proposed method in high-resolution global forecast and 4D-Var analysis experiments yields equally accurate results as the existing method, while reducing the cost of the iterative scheme.

show abstract

More accuracy with less precision

Cited by 23 publications

References 52 publications

Compressing atmospheric data into its real information content

Compressing atmospheric data into its real information content

Towards Strongly-coupled Ensemble Data Assimilation with Additional Improvements from Machine Learning

A fast converging and concise algorithm for computing the departure points in semi‐Lagrangian weather and climate models

Contact Info

Product

Resources

About