Accuracy of random-forest-based imputation of missing data in the presence of non-normality, non-linearity, and interaction

Hong, Shangzhi; Lynn, Henry

doi:10.1186/s12874-020-01080-1

Cited by 140 publications

(83 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, as the missing percentage grew, datasets imputed using missForest quickly became the most biased. This nding is consistent with previous evidence that implementing individual tree estimation through missForest could systematically lead to biased estimates, especially for non-normal, skewed data such as the count data that we had in our study [30]. Further, we found that the bias in imputed values became more aggravated as more missing values were introduced into the data.…”

Section: Discussionsupporting

confidence: 92%

“…For example, Waljee et al [4] have demonstrated the superiority of the local random forest method (e.g., missForest) in imputing missing laboratory values, while Hong & Lynn [5] have pointed out that the use of imputed variables from random forest-based approaches could lead to severely biased inference in a simulation study. Studies in other settings suggest that the results generated from multiple imputation are unbiased and could more closely mimic the true data [6,7].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Addressing missing values in routine health information system data: an evaluation of imputation methods using data from the Democratic Republic of the Congo during the COVID-19 pandemic

Feng

Hategeka

Grépin

2021

Preprint

View full text Add to dashboard Cite

Background : Poor data quality is limiting the greater use of data sourced from routine health information systems (RHIS), especially in low and middle-income countries. An important part of this issue comes from missing values, where health facilities, for a variety of reasons, miss their reports into the central system. Methods : Using data from the Health Management Information System (HMIS) and the advent of COVID-19 pandemic in the Democratic Republic of the Congo (DRC) as an illustrative case study, we implemented six commonly-used imputation methods using the DRC’s HMIS datasets and evaluated their performance through various statistical techniques, i.e., simple linear regression, segmented regression which is widely used in interrupted time series studies, and parametric comparisons through t-tests and non-parametric comparisons through Wilcoxon Rank-Sum tests. We also examined the performance of these six imputation methods under different missing mechanisms and tested their stability to changes in the data. Results : For regression analyses, there was no substantial difference found in the results generated from all methods except mean imputation and exclusion & interpolation when the RHIS dataset contained less than 20% missing values. However, as the missing proportion grew, machine learning methods such as missForest and k -NN started to produce biased estimates, and they were found to be also lack of robustness to minimal changes in data or to consecutive missingness. On the other hand, multiple imputation generated the overall most unbiased estimates and was the most robust to all changes in data. For comparing group means through t-tests, the results from mean imputation and exclusion & interpolation disagreed with the true inference obtained using the complete data, suggesting that these two methods would not only lead to biased regression estimates but also generate unreliable t-test results. Conclusions : We recommend the use of multiple imputation in addressing missing values in RHIS datasets. In cases necessary computing resources are unavailable to multiple imputation, one may consider seasonal decomposition as the next best method. Mean imputation and exclusion & interpolation, however, always produced biased and misleading results in the subsequent analyses, and thus their use in the handling of missing values should be discouraged. Keywords : Missing Data; Routine Health Information Systems (RHIS); Health Management Information System (HMIS); Health Services Research; Low and middle-income countries (LMICs); Multiple imputation

show abstract

Section: Discussionsupporting

confidence: 92%

Section: Introductionmentioning

confidence: 99%

Addressing missing values in routine health information system data: an evaluation of imputation methods using data from the Democratic Republic of the Congo during the COVID-19 pandemic

Feng

Hategeka

Grépin

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…However, the performance of these methods is not well stated. It depends on the presence and relevance of possible interaction effects and on the correlation structure of the data, and it could be quite poor when data are highly skewed [42][43][44]. It would be interesting to investigate the robustness to departure from the MAR assumption for multiple imputations approaches based on recursive partitioning.…”

Section: Discussionmentioning

confidence: 99%

Missing not at random in end of life care studies: multiple imputation and sensitivity analysis on data from the ACTION study

Carreras¹,

Miccinesi²,

Wilcock

et al. 2021

BMC Med Res Methodol

View full text Add to dashboard Cite

Background Missing data are common in end-of-life care studies, but there is still relatively little exploration of which is the best method to deal with them, and, in particular, if the missing at random (MAR) assumption is valid or missing not at random (MNAR) mechanisms should be assumed. In this paper we investigated this issue through a sensitivity analysis within the ACTION study, a multicenter cluster randomized controlled trial testing advance care planning in patients with advanced lung or colorectal cancer. Methods Multiple imputation procedures under MAR and MNAR assumptions were implemented. Possible violation of the MAR assumption was addressed with reference to variables measuring quality of life and symptoms. The MNAR model assumed that patients with worse health were more likely to have missing questionnaires, making a distinction between single missing items, which were assumed to satisfy the MAR assumption, and missing values due to completely missing questionnaire for which a MNAR mechanism was hypothesized. We explored the sensitivity to possible departures from MAR on gender differences between key indicators and on simple correlations. Results Up to 39% of follow-up data were missing. Results under MAR reflected that missingness was related to poorer health status. Correlations between variables, although very small, changed according to the imputation method, as well as the differences in scores by gender, indicating a certain sensitivity of the results to the violation of the MAR assumption. Conclusions The findings confirmed the importance of undertaking this kind of analysis in end-of-life care studies.

show abstract

“…Moreover, the algorithm is able to manage missing values which are common in clinical studies. The "on-the-fly-imputation" algorithm (Hong and Lynn, 2020) imputes data when it grows the forest handling interactions and non-linearity in the dataset.…”

Section: Methodsmentioning

confidence: 99%

Early prediction of in-hospital death of COVID-19 patients: a machine-learning model based on age, blood analyses, and chest x-ray score

Garrafa

Vezzoli

Ravanelli

et al. 2021

eLife

View full text Add to dashboard Cite

An early-warning model to predict in-hospital mortality on admission of COVID-19 patients at an emergency department (ED) was developed and validate using a Machine-Learning model. In total, 2782 patients were enrolled between March 2020 and December 2020, including 2106 patients (first wave) and 676 patients (second wave) in the COVID-19 outbreak in Italy. The first-wave patients were divided into two groups with 1474 patients used to train the model, and 632 to validate it. The 676 patients in the second wave were used to test the model. Age, 17 blood analytes and Brescia chest X-ray score were the variables processed using a Random Forests classification algorithm to build and validate the model. ROC analysis was used to assess the model performances. A web-based death-risk calculator was implemented and integrated within the Laboratory Information System of the hospital. The final score was constructed by age (the most powerful predictor), blood analytes (the strongest predictors were lactate dehydrogenase, D-dimer, Neutrophil/Lymphocyte ratio, C-reactive protein, Lymphocyte %, Ferritin std and Monocyte %), and Brescia chest X-ray score. The areas under the receiver operating characteristic curve obtained for the three groups (training, validating and testing) were 0.98, 0.83 and 0.78, respectively. The model predicts in-hospital mortality on the basis of data that can be obtained in a short time, directly at the ED on admission. It functions as a web-based calculator, providing a risk score which is easy to interpret. It can be used in the triage process to support the decision on patient allocation.

show abstract

Accuracy of random-forest-based imputation of missing data in the presence of non-normality, non-linearity, and interaction

Cited by 140 publications

References 18 publications

Addressing missing values in routine health information system data: an evaluation of imputation methods using data from the Democratic Republic of the Congo during the COVID-19 pandemic

Addressing missing values in routine health information system data: an evaluation of imputation methods using data from the Democratic Republic of the Congo during the COVID-19 pandemic

Missing not at random in end of life care studies: multiple imputation and sensitivity analysis on data from the ACTION study

Early prediction of in-hospital death of COVID-19 patients: a machine-learning model based on age, blood analyses, and chest x-ray score

Contact Info

Product

Resources

About