A novel approach for identifying and addressing case‐mix heterogeneity in individual participant data meta‐analysis

Abstract

Case‐mix heterogeneity across studies complicates meta‐analyses. As a result of this, treatments that are equally effective on patient subgroups may appear to have different effectiveness on patient populations with different case mix. It is therefore important that meta‐analyses be explicit for what patient population they describe the treatment effect. To achieve this, we develop a new approach for meta‐analysis of randomized clinical trials, which use individual patient data (IPD) from all trials to infer the treatment effect for the patient population in a given trial, based on direct standardization using either outcome regression (OCR) or inverse probability weighting (IPW). Accompanying random‐effect meta‐analysis models are developed. The new approach enables disentangling heterogeneity due to case mix from that due to beyond case‐mix reasons.

Assessing the impact of case-mix heterogeneity in individual participant data meta-analysis: Novel use of I2 statistic and prediction interval

“…This discussion makes use of counterfactual outcome theory and is inspired by our previous work and the work of Bareinboim and Pearl on non-transportability. 9,[16][17][18][19] Consider a meta-analysis of K randomized controlled trials (RCTs) evaluating the comparative effectiveness of two treatments (X ¼ 0, 1) on a dichotomous outcome Y. Assume that the randomization ratio is 1:1 across all studies.…”

“…To estimate RR(j, k), we further assume that the baseline covariate vector L contains all predictors of the outcome that are differentially distributed between studies. 9 Such an assumption implies that both Yð1 k Þ and Yð0 k Þ are independent of S given L. In practice, one important challenge could be that information on some baseline covariates is collected in some studies but not in the others. In such case, imputation methods might be needed to impute the missing covariate data (see for instance the methods proposed by Quartagno et al 20 and Resche-Rigon et al 21 ).…”

“…A drawback of this approach is that it comes with a high risk of undiagnosed extrapolation when patients in different studies have very different case mix. 9 In view of this, we alternatively consider an inverse probability weighting (IPW) approach:…”

“…being very large for some individuals. 9 These extreme weights can lead to a huge reduction in effective sample size, causing unstable results. For instance, on the probability scale, the IPW estimates for PrðYðx k Þ ¼ 1jS ¼ jÞ can be larger than 1 for certain values of j and k due to extreme weights.…”

“…7 Moreover, by focusing on subgroup effects, these approaches deliver treatment effects whose magnitude may fail to be comparable with that observed in the original (unadjusted) meta-analysis. 8,9 In conventional meta-analysis, heterogeneity assessment is often supported by two statistics-the I 2 statistic and the prediction interval. 10,11 Originally proposed by Higgins et al, 10 the I 2 statistic quantifies the proportion of the total heterogeneity that is not due to sampling error in the standard random-effect meta-analyses.…”