A cautionary tale on using imputation methods for inference in matched-pairs design

Abstract

Motivation: Imputation procedures in biomedical fields have turned into statistical practice, since further analyses can be conducted ignoring the former presence of missing values. In particular, non-parametric imputation schemes like the random forest or a combination with the stochastic gradient boosting have shown favorable imputation performance compared to the more traditionally used MICE procedure. However, their effect on valid statistical inference has not been analyzed so far. This paper closes this gap by investigating their validity for inferring mean differences in incompletely observed pairs while opposing them to a recent approach that only works with the given observations at hand. Results: Our findings indicate that machine learning schemes for (multiply) imputing missing values may inflate type-I-error or result in comparably low power in small to moderate matched pairs, even after modifying the test statistics using Rubin's multiple imputation rule. In addition to an extensive simulation study, an illustrative data example from a breast cancer gene study has been considered. Availability: The corresponding R-code can be accessed through the authors and the gene expression data can be downloaded at www.gdac.broadinstitute.org

Ranking procedures for repeated measures designs with missing data: Estimation, testing and asymptotic theory

“…The results for the newly proposed MCTP which uses all-available information and the MCTP using only complete cases are comparable. As expected (van Buuren 28 , Ramosaj et al 29 ), the simple median imputation yields in some scenarios with many missings extremly inflated type-I error rates and is therefore not recommend. Note that data has been generated under MCAR and MAR assumptions for this comparison.…”

Ranks and Pseudo‐ranks—Surprising Results of Certain Rank Tests in Unbalanced Designs

“…This is, however, not true for modifications of these procedures. For example, if unequal numbers of replications within the different blocks and treatments are performed or if missing values occur in an unbalanced pattern (see also Ramosaj et al , 2018, for additional difficulties that may appear in this context), then also surprising results for these tests are possible.…”

Is there a role for statistics in artificial intelligence?

“…Directions for future developments include personalized medicine aiming at tailoring treatments to patient subgroups (strata) or even individual patients (Hamburg and Collins 2010;Blasiak et al 2020;Schork 2019). Furthermore, official statistics uses AI methods for classification as well as for recognition, estimation and/or imputation of relevant characteristic values of statistical units (Beck et al 2018;Ramosaj and Pauly 2019b;Ramosaj et al 2020;UNECE 2020;Thurow et al 2021). In economics and econometrics, AI methods are also applied and further developed, for example, to draw conclusions about macroeconomic developments from large amounts of data on individual consumer behavior (McCracken and Ng 2016;Ng 2018).…”