In this paper we shall present the derivation of two new forms of the Kalman filter equations; the first is for a pure lognormally distributed random variable, while the second set of Kalman filter equations will be for a combination of Gaussian and lognormally distributed random variables. We shall show that the appearance is similar to that of the Gaussian based equations, but that the analysis state is a multivariate median and not the mean. We also show results of the mixed distribution Kalman filter with the Lorenz 1963 model with lognormal errors for the background and observations of the ɀ component, and compare them to analysis results from a traditional Gaussian based extended Kalman filter and show that under certain circumstances the new approach produces more accurate results.

An important assumption made in most variational, ensemble, and hybrid‐based data assimilation systems is that all minimized errors are Gaussian random variables. A theory developed at the Cooperative Institute for Research in the Atmosphere enables for the Gaussian assumption for the different types of errors to be relaxed to a lognormally distributed random variable. While this is a first step toward using more consistent distributions to model the errors involved in numerical weather/ocean prediction, we still need to be able to identify when we need to assign a lognormal distribution in a mixed Gaussian‐lognormal approach. In this paper, we present some machine learning techniques and experiments with the Lorenz 63 model. Using these machine learning techniques, we show detection of non‐Gaussian distributions can be done using two methods: a support vector machine and a neural network. This is done by training past data to classify (1) differences with the distribution statistics (means and modes) and (2) the skewness of the probability density function.

The assumption that variables, and their errors, are Gaussian distributed is commonplace in areas such as numerical weather prediction and modeling. Research such as that undertaken by Perron and Sura (2013) has shown that this assumption is generally false for atmospheric variables, and that Gaussian variables in the atmosphere are rare. The aforementioned statement was based on a 62 year long project from daily data taken from the National Centers for Environmental Prediction and the National Center for Atmospheric Research, using the Reanalysis I Project data set. Given this evidence, the need to be able to relax the Gaussian assumption for the errors involved in the data assimilation schemes becomes quite important if the analysis error is to be minimized. By doing so, we may deliver an improvement in the subsequent forecast.Most of the current formulations of data assimilation, for example, variational methods such as 3DVar and 4DVar (which are based upon Bayes theorem Fletcher, 2017), and ensemble methods such as the Ensemble Kalman Filter (EnKF;Evensen & Van Leeuwen, 1996), the (local) Ensemble Transform Kalman Filter ((L)ETKF) Ott et al. (2004); Wang and Bishop (2003), which are based upon a control theory/weighted least squares approach using ensemble members to approximate the analysis mean and covariances, and the Maximum Likelihood Ensemble Kalman Filter, Zupanski (2005), which uses the Kalman filter equations combined with the 3DVar cost function, all assume that the errors involved are Gaussian distributed. Also in linear state estimation, the initial condition x 0 is also assumed to be Gaussian distributed. Other papers that look into non-Gaussian data assimilation methods are local particle filters, van Leeuwen et al. (2019), andLeeuwen (2014) who looked into Gaussian anamorphosis on the EnKF. However, at the Cooperative Institute for Research in the Atmosphere (CIRA) at Colorado State University, a theory has been developed and tested, that allows for the Gaussian assumption for the distribution of the errors to be relaxed to a lognormal distribution. In Fletcher and Zupanski (2006a), the theory is presented for the case

The assumption of a Gaussian distribution is widely used in remote sensing retrievals and data assimilation for numerical weather prediction. Since many geophysical variables follow a log‐normal distribution rather than a Gaussian distribution, a mixed log‐normal and Gaussian distribution data assimilation scheme is implemented in the Cooperative Institute for Research in the Atmosphere (CIRA) one‐dimensional optimal estimation (C1DOE) retrieval system to model the background errors associated with the temperature as a Gaussian and those with respect to the mixing ratio as log‐normal. The new mixed distribution is compared against the traditional Gaussian configuration and a logarithmic transformation for the water‐vapour mixing ratio for two situations: (i) synthetic brightness temperatures generated from a log‐normal distribution and (ii) Advanced Microwave Sounding Unit (AMSU) observations from the month of September 2005 over the west Pacific, where a log‐normal signal for moisture had previously been detected. It is shown for Case 1 that, given a consistent a priori state for a log‐normal distribution, the log‐normal distribution based retrieval is the best at inverting the true state back. For Case 2, the final innovations are smallest for the mixed distribution scheme. The retrieval values are compared against the Microwave Surface and Precipitation Products System (MSPPS) and the log‐normal approach is consistently closer to the values compared with these other two approaches.