Influence diagnostics and outlier detection for meta‐analysis of diagnostic test accuracy

Abstract

Meta‐analyses of diagnostic test accuracy (DTA) studies have been gaining prominence in research in clinical epidemiology and health technology development. In these DTA meta‐analyses, some studies may have markedly different characteristics from the others and potentially be inappropriate to include. The inclusion of these “outlying” studies might lead to biases, yielding misleading results. In addition, there might be influential studies that have notable impacts on the results. In this article, we propose Bayesian methods for detecting outlying studies and their influence diagnostics in DTA meta‐analyses. Synthetic influence measures based on the bivariate hierarchical Bayesian random effects models are developed because the overall influences of individual studies should be simultaneously assessed by the two outcome variables and their correlation information. We propose four synthetic measures for influence analyses: (a) relative distance, (b) standardized residual, (c) Bayesian p‐value, and (d) influence statistic on the area under the summary receiver operating characteristic curve. We also show that conventional univariate Bayesian influential measures can be applied to the bivariate random effects models, which can be used as marginal influential measures. Most of these methods can be similarly applied to the frequentist framework. We illustrate the effectiveness of the proposed methods by applying them to a DTA meta‐analysis of ultrasound in screening for vesicoureteral reflux among children with urinary tract infections.

Quantitative evidence synthesis: a practical guide on meta-analysis, meta-regression, and publication bias tests for environmental sciences

“…Unfortunately, in our survey, only 37% of environmental meta-analyses (27 out of 73) conducted sensitivity analysis. There are two general and interrelated ways to conduct sensitivity analyses [60,76,77]. The first one is to take out influential studies (e.g., outliers) and re-run meta-analytic and meta-regression models.…”

Sensitivity analysis with iterative outlier detection for systematic reviews and meta‐analyses

“…Although outlier detection has been a well-studied topic in the statistical community, relatively few efforts have been especially paid to the field of meta-analysis. [28][29][30][31][32][33][34][35] Hedges and Olkin 36 (Chapter 12) presented several early efforts for detecting outliers in a meta-analysis, but they primarily considered the common-effect setting, where the studies are assumed to share a common true treatment effect. The leave-one-study-out procedure by Viechtbauer and Cheung 37 in the random-effects setting is perhaps the most widely used tool for detecting outliers in the current practice of meta-analysis.…”

“…Although outlier detection has been a well‐studied topic in the statistical community, relatively few efforts have been especially paid to the field of meta‐analysis 28‐35 . Hedges and Olkin 36 (Chapter 12) presented several early efforts for detecting outliers in a meta‐analysis, but they primarily considered the common‐effect setting, where the studies are assumed to share a common true treatment effect.…”

Bayesian model-based outlier detection in network meta-analysis

“…binomial for binary data) rather than relying on normal approximations. However, outlier detection in the Bayesian framework has not been sufficiently explored, with exception of one method using Bayesian p-values as detection tool in a meta-analysis for DTAs 12 and a second one in a network meta-analysis mainly based on arm-based models for continuous outcome 13 . A comprehensive assessment of outlying studies in the network should not only focus on the statistical detection of extreme effect measures (or variances) of the studies included, rather, it should try to understand the causes behind it through careful appraisals of the characteristics of each included study.…”