Comparing methods for handling missing covariates in meta‐regression

Abstract

Meta-analysts often encounter missing covariate values when estimating metaregression models. In practice, ad hoc approaches involving data deletion have been widely used. The current study investigates the performance of different methods for handling missing covariates in meta-regression, including complete-case analysis (CCA), shifting-case analysis (SCA), multiple imputation (MI), and full information maximum likelihood (FIML), assuming missing at random mechanism. According to the simulation results, we advocate the use of MI and FIML than CCA and SCA approaches in practice. In addition, we cautiously note the challenges and potential advantages of using MI in the meta-analysis context.

Missingness mechanism messiness in meta‐analysis: A response to “The importance of plausible data‐generating mechanisms in simulation studies”

“…We acknowledged that the performance of complete‐case analysis (CCA) depends on the particular form of the missingness model, rather than on a broader classification of missingness mechanisms (see the article 1 p. 15–16). Specifically, whether the probability of a covariate being missing is a function of the outcome variable (i.e., the effect size in meta‐analytic data) and the strength of the relationship between the two; these two factors will influence the validity of CCA's results.…”

“…In our view, the plausibility of certain missing data mechanisms is infeasible to definitively identify and might differ depending on researchers' own assumptions, research topics, fields, and contexts. In the article 1 (p. 3), we hypothesized that “study characteristics (in meta‐analytic data) are likely to be missing as a function of other factors (i.e., other study designs, study discipline, or even effect sizes) than to be missing completely at random (MCAR)” 2,3 and “this (MAR) scenario is what we focus on in the current study.” This offered us a starting point for this line of research and was based on the authors' applied meta‐analytic experience and discussions with other meta‐analysis researchers in the fields of education and psychology.…”

“…In this response, we discuss our assumptions, views of the plausibility of the missingness mechanisms that we modeled and address the reviewer's feedback about our assumed missingness mechanism and the missingness model that we used in our study. 1 All simulation studies are inherently limited in their generalizability. Missingness mechanisms and the types of models used to generate the missing data can affect the performance of missing data handling methods and the results only apply to the particular conditions examined in the study.…”

“…We thank Dr. White for his thoughtful commentary, for his interest in our research and for sharing concerns which support the future direction of our ongoing research. In this response, we discuss our assumptions, views of the plausibility of the missingness mechanisms that we modeled and address the reviewer's feedback about our assumed missingness mechanism and the missingness model that we used in our study 1 …”

“…We acknowledge that the missingness mechanism and the specific type of missingness model assumed in the article 1 were restricted to a single scenario which limits the generalizability of the simulation results. We discussed this as an important limitation of this study in the article 1 (p. 18). Because a strong missing at random (MAR) mechanism represents only one of many possible scenarios, we intended to present the contribution of our work as a starting point that leaves open other opportunities for research, including testing missing data handling methods under different assumptions for missing covariates in meta‐analysis.…”