Missing covariates is a common issue when fitting meta-regression models. Standard practice for handling missing covariates tends to involve one of two approaches. In a complete-case analysis, effect sizes for which relevant covariates are missing are omitted from model estimation. Alternatively, researchers have employed the so-called "shifting units of analysis" wherein complete-case analyses are conducted on only certain subsets of relevant covariates. In this article, we clarify conditions under which these approaches generate unbiased estimates of regression coefficients. We find that unbiased estimates are possible when the probability of observing a covariate is completely independent of effect sizes.When that does not hold, regression coefficient estimates may be biased. We study the potential magnitude of that bias assuming a log-linear model of missingness and find that the bias can be substantial, as large as Cohen's d = 0.4-0.8 depending on the missingness mechanism. K E Y W O R D Scomplete-case analysis, meta-regression, missing data, shifting units of analysis HighlightsMissing covariates are a common problem when conducting meta-regressions. A common practice for meta-regression analyses has been to ignore effects for which covariates are missing. However, a vast statistical literature suggests that analyses that ignore missing data can only provide accurate estimates of relevant quantitites under certain conditions. In this article, we examine conditions under which ignoring missing covariates in a meta-regression can still lead to unbiased estimation of regression coefficients. We also investigate the possible magnitude and sources of bias when those conditions do not hold. Our findings highlight that substantial bias can be induced by ignoring missing data in a meta-regression. | INTRODUCTIONMeta-regression is a useful tool for studying important sources of variation between effects in a meta-analysis. 1,2 Analyses of these models in the absence of missing data have been studied thoroughly in the literature. [3][4][5][6][7] However, it is common for meta-analytic datasets to be missing data. 8 In the context of meta-regression,

The impact of online math programs depends on its implementation, especially in vulnerable populations from developing countries. An existing online platform was adapted, at the request of the Chilean Ministry of Education, to exclusively include exercises previously designed and tested by a paper-based government program for elementary school. We carried out a cluster-randomized controlled trial (RCT) with 50 fourth grade classrooms. Treatment classrooms used the platform in a weekly 90-min math session. Due to a social instability outbreak in the country, a large unexpected disruption with huge absenteeism occurred in the second half of the semester, which turned this study into a unique opportunity to explore the robustness of the platform’s effects on students’ learning. Using multiple imputation and multilevel models, we found a statistically significant effect size of 0.13, which corresponds to two extra months of learning. This effect is meaningful for four reasons. First, it has double the effect of the paper-based version. Second, it was achieved during one semester only. Third, is half that obtained with the platform for a complete year with its own set of exercises and with two sessions per week instead of one. Fourth, it was attained in a semester with a lot of absenteeism.

Objectives In this tutorial, we examine methods for exploring missingness in a dataset in ways that can help to identify the sources and extent of missingness, as well as clarify gaps in evidence. Methods Using raw data from a meta-analysis of substance abuse interventions, we demonstrate the use of exploratory missingness analysis (EMA) including techniques for numerical summaries and visual displays of missing data. Results These techniques examine the patterns of missing covariates in meta-analysis data and the relationships among variables with missing data and observed variables including the effect size. The case study shows complex relationships among missingness and other potential covariates in meta-regression, highlighting gaps in the evidence base. Conclusion Meta-analysts could often benefit by employing some form of EMA as they encounter missing data.