Accounting for Correlated Observation Error in a Dual-Formulation 4D Variational Data Assimilation System

Abstract

Appropriate specification of the error statistics for both observational data and short-term forecasts is necessary to produce an optimal analysis. Observation error stems from instrument error, forward model error, and error of representation. All sources of observation error, particularly error of representation, can lead to nonzero correlations. While correlated forecast error has been accounted for since the early days of atmospheric data assimilation, observation error has typically been treated as uncorrelated until relatively recently. Thinning, averaging, and/or inflation of the assigned observation error variance have been employed to compensate for unaccounted error correlations, especially for high-resolution satellite data. In this study, the benefits of accounting for nonzero vertical (interchannel) correlation for both the Advanced Technology Microwave Satellite (ATMS) and Infrared Atmospheric Sounding Interferometer (IASI) in the NRL Atmospheric Variational Data Assimilation System-Accelerated Representer (NAVDAS-AR) are assessed. The vertical observation error covariance matrix for the ATMS and IASI instruments was estimated using the Desroziers method. The results suggest lowering the assigned error variance and introducing strong correlations, especially in the moisture-sensitive channels. Strong positive impact on forecast skill (verified against both the ECMWF analyses and high-quality radiosonde data) is shown in both the ATMS and IASI instruments. Additionally, the convergence of the iterative solver in NAVDAS-AR can be improved by small modifications to the observation error covariance matrices, resulting in further reduction in RMS error.

The conditioning of least‐squares problems in variational data assimilation

“…This comes from the restriction on the form of S when additional assumptions are made on R and H and does not generalise to the other results presented in this work. We also observe that the lower bound (22) has an explicit dependence on the number of observations, p, meaning that (S) increases with p. This was studied in detail by Haben. 16 Additionally, the ratio 2 b 2 o appears in both bounds, meaning that the discussion following the result of Corollary 2 also applies to the result of Theorem 7.…”

“…Correlated observation error statistics have been diagnosed for certain observation types (e.g., see other works [20][21][22][23][24][25][26][27] ), although there are problems associated with their use. In particular, the methods used to diagnose observation error covariance matrices are imperfect, and the quality of these estimates is unclear.…”

“…There is significant research investigating spatial correlations, 20,23,27,51 but much current work concerns the practical implementation of interchannel correlations for satellite observations. 3,6,9,21,22,26 Although the theory presented in Section 3 applies directly to the case of interchannel correlations, it would be of interest to extend our numerical testing to the interchannel covariance case. In particular, practical experiments have revealed that the minimum eigenvalue of the observation error correlation matrix is important for the conditioning of the Hessian in the case of interchannel correlations, 3,9 which coincides with the theoretical and experimental results presented in this work.…”

Technical note: Assessment of observation quality for data assimilation in flood models

“…The diagnostic has been applied to operational numerical weather prediction (NWP) settings to estimate observation uncertainties (Stewart et al, 2014;Waller et al, 2016a, c;Cordoba et al, 2017). The use of these estimated statistics in NWP results in a more accurate analysis and improvements in objec-tively measured forecast skill (Weston et al, 2014;Bormann et al, 2016;Campbell et al, 2017).…”

Modelling spatially correlated observation errors in variational data assimilation using a diffusion operator on an unstructured mesh

“…For radiances, it is customary to separate the vertical (or inter‐channel) correlations from the horizontal spatial correlations. In recent years, substantial progress has been made in representing inter‐channel error correlations (Bormann et al ; Bormann and Bauer, ; Stewart et al ; Weston et al ; Waller et al ; Campbell et al ). The size of the matrices required to represent these correlations is rather small (less than 10 3 rows or columns), which makes them straightforward to handle computationally using, for example, Cholesky decomposition.…”