Automated data-driven and stochastic imputation method

Abstract

Machine learning algorithms may have difficulty processing datasets with missing values. Identifying and replacing missing values is necessary before modeling the prediction for missing data. However, studies have shown that uniformly compensating for missing values in a dataset is impossible, and no imputation technique fits all datasets. This study presents an Automated and data-driven Stochastic Imputer (ASI). The proposed ASI is based on automated distribution detection and estimation of the imputed value by stochastic sampling with controlled error probability. The significant advantage of this method is the use of a data-driven approximation ratio for the stochastic sampling, which bound the samples to be, at most, one standard deviation from the original distribution. The ASI performance was compared to traditional deterministic and stochastic imputation algorithms over seven datasets. The results showed that ASI succeeded in 61.5% of cases compared to other algorithms, and its performance can be improved by controlling the sampling error probability.