An Ensemble Method for Missing Data of Environmental Sensor Considering Univariate and Multivariate Characteristics

Abstract

With rapid urbanization, awareness of environmental pollution is growing rapidly and, accordingly, interest in environmental sensors that measure atmospheric and indoor air quality is increasing. Since these IoT-based environmental sensors are sensitive and value reliability, it is essential to deal with missing values, which are one of the causes of reliability problems. Characteristics that can be used to impute missing values in environmental sensors are the time dependency of single variables and the correlation between multivariate variables. However, in the existing method of imputing missing values, only one characteristic has been used and there has been no case where both characteristics were used. In this work, we introduced a new ensemble imputation method reflecting this. First, the cases in which missing values occur frequently were divided into four cases and were generated into the experimental data: communication error (aperiodic, periodic), sensor error (rapid change, measurement range). To compare the existing method with the proposed method, five methods of univariate imputation and five methods of multivariate imputation—both of which are widely used—were used as a single model to predict missing values for the four cases. The values predicted by a single model were applied to the ensemble method. Among the ensemble methods, the weighted average and stacking methods were used to derive the final predicted values and replace the missing values. Finally, the predicted values, substituted with the original data, were evaluated by a comparison between the mean absolute error (MAE) and the root mean square error (RMSE). The proposed ensemble method generally performed better than the single method. In addition, this method simultaneously considers the correlation between variables and time dependence, which are characteristics that must be considered in the environmental sensor. As a result, our proposed ensemble technique can contribute to the replacement of the missing values generated by environmental sensors, which can help to increase the reliability of environmental sensor data.

Missing-value Imputation of Environment Sensors Using Multilayer Stacking with Scoring Method

“…In this study, imputation was performed from different angles, according to the de ned sensor imputation types [24]. We considered missing situations using case types that may occur in IoT-based environmental sensor modules and determined which node model affects the results.…”

Methods of Measuring Air Pollution in Cities and Correlation of Air Pollutant Concentrations

“…These systems are necessary to monitor the manner in which alternative and sustainable energy sources affect air quality when deployed. Also relevant is the research which aims to fill the gaps caused by the inevitable malfunction of IoT systems and sensors [12], which results in missing data about air quality parameters and can skew data one way or the other. Such methods are important when reliability and consistency of sensor readings is of utmost importance.…”

Methodological approaches for imputing missing data into monthly flows series

“…Other studies can be highlighted, especially related to medicine and health (Camargos et al, 2011;Carreras et al, 2021;Khan et al, 2021;Nunes, 2007;Payrovnaziri et al, 2021), air pollution (Choi et al, 2021;Ghazali et al, 2021;Pinto, 2013), engineering, mainly civil and traffic (Abdelgawad et al, 2015;Jiang et al, 2021), meteorology (Afrifa-Yamoah et al, 2020Bier and Ferraz, 2017;Costa et al, 2021;Ferrari and Ozaki, 2014;García-Peña et al, 2014), agriculture (Jiao et al, 2016;Nishina et al, 2017;Swenson, 2014), energy (Barbosa et al, 2018;Pelisson, 2021) and education (Vinha and Laros, 2018).…”