Application of imputation methods for missing values of PM₁₀ and O₃ data: Interpolation, moving average and K-nearest neighbor methods

Abstract

Background: PIn air quality studies, it is very often to have missing data due to reasons such as machine failure or human error. The approach used in dealing with such missing data can affect the results of the analysis. The main aim of this study was to review the types of missing mechanism, imputation methods, application of some of them in imputation of missing of PM10 and O3 in Tabriz, and compare their efficiency. Methods: Methods of mean, EM algorithm, regression, classification and regression tree, predictive mean matching (PMM), interpolation, moving average, and K-nearest neighbor (KNN) were used. PMM was investigated by considering the spatial and temporal dependencies in the model. Missing data were randomly simulated with 10, 20, and 30% missing values. The efficiency of methods was compared using coefficient of determination (R2 ), mean absolute error (MAE) and root mean square error (RMSE). Results: Based on the results for all indicators, interpolation, moving average, and KNN had the best performance, respectively. PMM did not perform well with and without spatio-temporal information. Conclusion: Given that the nature of pollution data always depends on next and previous information, methods that their computational nature is based on before and after information indicated better performance than others, so in the case of pollutant data, it is recommended to use these methods.

Prediction of daily fine particulate matter (PM2.5) concentration in Aksaray, Turkey: Temporal variation, meteorological dependence, and employing artificial neural network

“…The bias of multivariate estimations like correlation or regression coefficients is one of the drawbacks of the linear moving average imputation method. In general, there is no link between the values imputed by the mean of the variables and the other variables 42 …”

Research on Missing Value Imputation to Improve the Validity of Air Quality Data Evaluation on the Qinghai-Tibetan Plateau

“…In contrast, machine learning and deep learning approaches often yield superior imputation results but typically necessitate extended imputation durations compared to statistical methods. Concurrently, traditional machine learning approaches, encompassing K-Nearest Neighbor, fuzzy methods, decision trees, support vectors, and other models, have been integrated into the repertoire of techniques for addressing missing values [29][30][31]. A case in point is the work of Honghai et al, where Support Vector Machine (SVM) regression was employed to estimate missing conditional attribute values, illustrating the efficacy of machine learning in enhancing data completeness, but not with large datasets [32].…”

Trend analysis of Humidex as a heat discomfort index using Mann-Kendall and Sen’s slope estimator statistical tests

“…Urban development and expansion have caused changes in both climatic and atmospheric conditions in recent years (5). These changes have affected the stability of the natural environment and the health of people especially those who live in urban areas (6).…”