Pattern-based Comparative Analysis of Techniques for Missing Value Imputation

Sadhu, Ambika; Soni, Rishabh; Mishra, Mridul

doi:10.1109/iccca49541.2020.9250825

Cited by 3 publications

(3 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Datawig makes predictions for missing Likert-type items by using gradient-boosted trees, a type of ensemble learning method that builds multiple decision trees and combines their predictions to make a final prediction (Sadhu et al, 2020). When predicting missing Likert-type items, the gradient-boosted trees learn the relationships between the observed values and the target variable, taking into account the ordinal information.…”

Section: Empirical Examplementioning

confidence: 99%

Deep Learning Imputation for Asymmetric and Incomplete Likert-Type Items

Collier¹,

Kong²,

Soyoye³

et al. 2023

Journal of Educational and Behavioral Statistics

View full text Add to dashboard Cite

Asymmetric Likert-type items in research studies can present several challenges in data analysis, particularly concerning missing data. These items are often characterized by a skewed scaling, where either there is no neutral response option or an unequal number of possible positive and negative responses. The use of conventional techniques, such as discriminant analysis or logistic regression imputation, for handling missing data in asymmetric items may result in significant bias. It is also recommended to exercise caution when employing alternative strategies, such as listwise deletion or mean imputation, because these methods rely on assumptions that are often unrealistic in surveys and rating scales. This article explores the potential of implementing a deep learning-based imputation method. Additionally, we provide access to deep learning-based imputation to a broader group of researchers without requiring advanced machine learning training. We apply the methodology to the Wilmington Street Participatory Action Research Health Project.

show abstract

Section: Empirical Examplementioning

confidence: 99%

Deep Learning Imputation for Asymmetric and Incomplete Likert-Type Items

Collier¹,

Kong²,

Soyoye³

et al. 2023

Journal of Educational and Behavioral Statistics

View full text Add to dashboard Cite

show abstract

“…The k-nearest neighbor (kNN) algorithm is an instance-based estimation procedure using the ‘k’-nearest neighbor values of the feature. The value that is imputed is generally the mean of the k-nearest neighbor values, and they can also be modified accordingly [ 23 ]. The kNN imputation method has been successfully applied in the real-time processing of data due to its simplicity and high accuracy [ 18 , 24 ].…”

Section: Introductionmentioning

confidence: 99%

Binned Data Provide Better Imputation of Missing Time Series Data from Wearables

Chakrabarti¹,

Biswas²,

Karnani

et al. 2023

Sensors

View full text Add to dashboard Cite

The presence of missing values in a time-series dataset is a very common and well-known problem. Various statistical and machine learning methods have been developed to overcome this problem, with the aim of filling in the missing values in the data. However, the performances of these methods vary widely, showing a high dependence on the type of data and correlations within the data. In our study, we performed some of the well-known imputation methods, such as expectation maximization, k-nearest neighbor, iterative imputer, random forest, and simple imputer, to impute missing data obtained from smart, wearable health trackers. In this manuscript, we proposed the use of data binning for imputation. We showed that the use of data binned around the missing time interval provides a better imputation than the use of a whole dataset. Imputation was performed for 15 min and 1 h of continuous missing data. We used a dataset with different bin sizes, such as 15 min, 30 min, 45 min, and 1 h, and we carried out evaluations using root mean square error (RMSE) values. We observed that the expectation maximization algorithm worked best for the use of binned data. This was followed by the simple imputer, iterative imputer, and k-nearest neighbor, whereas the random forest method had no effect on data binning during imputation. Moreover, the smallest bin sizes of 15 min and 1 h were observed to provide the lowest RMSE values for the majority of the time frames during the imputation of 15 min and 1 h of missing data, respectively. Although applicable to digital health data, we think that this method will also find applicability in other domains.

show abstract

“…Several studies have published the missing data problem in the ML domain [7], [11]- [14]. However, these studies have been dispersed among different journals and conference proceedings.…”

Section: Introductionmentioning

confidence: 99%

Systematic Review of Using Machine Learning in Imputing Missing Values

et al. 2022

View full text Add to dashboard Cite

Missing data are a universal data quality problem in many domains, leading to misleading analysis and inaccurate decisions. Much research has been done to investigate the different mechanisms of missing data and the proper techniques in handling various data types. In the last decade, machine learning has been utilized to replace conventional methods to address the problem of missing values more efficiently. By studying and analyzing recently proposed methods using machine learning approaches, vital adoptions in accuracy, performance, and time consumed can be highlighted. This study aimed to help data analysts and researchers address the limitations of machine learning imputation methods by conducting a systematic literature review to provide a comprehensive overview of using such methods to impute missing values. Novel proposed machine learning approaches used for data imputation are analyzed and summarized to assist researchers in selecting a proper machine learning method based on several factors and settings. The review was performed on research studies published between 2016 and 2021 on adopting machine learning to impute missing values, focusing on their strengths and limitations. A total of 684 research articles from various scientific databases were analyzed using search engines, and 94 of them were selected as primary studies. Finally, several recommendations were given to guide future researchers in applying machine learning to impute missing values.INDEX TERMS Systematic literature review, data imputation, data mining, missingness, data preprocessing, data quality.

show abstract

Pattern-based Comparative Analysis of Techniques for Missing Value Imputation

Cited by 3 publications

References 5 publications

Deep Learning Imputation for Asymmetric and Incomplete Likert-Type Items

Deep Learning Imputation for Asymmetric and Incomplete Likert-Type Items

Binned Data Provide Better Imputation of Missing Time Series Data from Wearables

Systematic Review of Using Machine Learning in Imputing Missing Values

Contact Info

Product

Resources

About