Statistical methods for detecting outlying and influential studies in meta-analysis of diagnostic test accuracy studies

Negeri, Zelalem; Beyene, Joseph

doi:10.1177/0962280219852747

Cited by 11 publications

(25 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This multivariate studentized residuals were also proposed by Negeri and Beyene 14 for the bivariate random-effects model of DTA meta-analysis. However, as noted above, there are not missing outcomes in DTA meta-analysis, whereas network meta-analysis models usually involve many missing outcomes.…”

Section: Comparison-specific Studentized Residualmentioning

confidence: 82%

“…Negeri and Beyene 14 proposed an alternative influence measure that directly assesses the change in the estimate of heterogeneity covariance matrix

\overset{Ψ}{true}

using the LOTOCV framework. The proposed measure was the ratio of generalized variances of the estimates of Ψ for the leave‐one‐trial‐out dataset and the all trial dataset.…”

Section: Leave‐one‐trial‐out Cross‐validated Influential Measuresmentioning

confidence: 99%

“…For frequentist inference methods of the conventional univariate meta‐analysis, several useful influence diagnosis methods were developed by Viechtbauer and Chueng 12 and Gumedze and Jackson 13 . Also, Negeri and Beyene 14 generalized their methods to the bivariate random‐effects model for meta‐analyses of diagnostic test accuracy (DTA) studies; but the DTA meta‐analyses involve only two outcome variables, and complete observations are usually obtained for both of the two outcome variables, sensitivity and specificity 15 . In network meta‐analysis, the dimension of the multivariate model is usually not small (for the illustrative example of antihypertensive drugs 16 in Sections 2 and 6, the dimension was 7), and most of the components of outcome variables for the multivariate model are “missing” (not defined) because most individual trials usually involve only two to five arms.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Outlier detection and influence diagnostics in network meta‐analysis

Noma

Gosho

Ishii

et al. 2020

Research Synthesis Methods

View full text Add to dashboard Cite

Network meta-analysis has been gaining prominence as an evidence synthesis method that enables the comprehensive synthesis and simultaneous comparison of multiple treatments. In many network meta-analyses, some of the constituent studies may have markedly different characteristics from the others, and may be influential enough to change the overall results. The inclusion of these "outlying" studies might lead to biases, yielding misleading results. In this article, we propose effective methods for detecting outlying and influential studies in a frequentist framework. In particular, we propose suitable influence measures for network meta-analysis models that involve missing outcomes and adjust the degree of freedoms appropriately. We propose three influential measures by a leave-one-trial-out cross-validation scheme: (1) comparison-specific studentized residual, (2) relative change measure for covariance matrix of the comparative effectiveness parameters, (3) relative change measure for heterogeneity covariance matrix. We also propose (4) a model-based approach using a likelihood ratio statistic by a mean-shifted outlier detection model. We illustrate the effectiveness of the proposed methods via applications to a network meta-analysis of antihypertensive drugs. Using the four proposed methods, we could detect three potential influential trials involving an obvious outlier that was retracted because of data falsifications. We also demonstrate that the overall results of comparative efficacy estimates and the ranking of drugs were altered by omitting these three influential studies. K E Y W O R D S contrast-based model, influence diagnostics, multivariate meta-analysis, network meta-analysis, outlier detection 1 | INTRODUCTION Network meta-analysis has been gaining prominence as an evidence synthesis method that enables the comprehensive synthesis and simultaneous comparison of multiple treatments. The methodology can synthesize both direct and indirect evidence for all treatment comparisons of interest, and provides estimates of comparative efficacies among treatments, even when no direct comparison evidence exists for some of the included treatments. 1,2 In network meta-analysis, there are often systematic heterogeneities among the synthesized studies, for example, study designs, participant characteristics, regions, sites, treatment administration, interventions, outcome definitions, and so on. Therefore, heterogeneity of the

show abstract

Section: Comparison-specific Studentized Residualmentioning

confidence: 82%

“…Negeri and Beyene 14 proposed an alternative influence measure that directly assesses the change in the estimate of heterogeneity covariance matrix

\overset{Ψ}{true}

using the LOTOCV framework. The proposed measure was the ratio of generalized variances of the estimates of Ψ for the leave‐one‐trial‐out dataset and the all trial dataset.…”

Section: Leave‐one‐trial‐out Cross‐validated Influential Measuresmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Outlier detection and influence diagnostics in network meta‐analysis

Noma

Gosho

Ishii

et al. 2020

Research Synthesis Methods

View full text Add to dashboard Cite

show abstract

“…4 They proposed a bivariate residual-based diagnostic method and a test-based method using a mean-shift outlier model within the frequentist framework. 13 Besides, Bayesian influence diagnostic methods are another effective approaches for these problems. 14 In particular, the influence diagnostic methods of Carlin and Louis 14 have been widely applied to various statistical problems as useful tools for outlier detections.…”

Section: Introductionmentioning

confidence: 99%

“…For outlier detection in DTA meta‐analysis, several exploratory methods and graphical tools have been discussed in the literature, but they have limitations because of their heuristic and subjective approaches . Recently, to address these issues, Negeri and Beyene proposed more objective approaches based on the Reitsma's frequentist bivariate random effects model . They proposed a bivariate residual‐based diagnostic method and a test‐based method using a mean‐shift outlier model within the frequentist framework .…”

Section: Introductionmentioning

confidence: 99%

Influence diagnostics and outlier detection for meta‐analysis of diagnostic test accuracy

Matsushima

Noma

Yamada

et al. 2019

Research Synthesis Methods

View full text Add to dashboard Cite

Meta‐analyses of diagnostic test accuracy (DTA) studies have been gaining prominence in research in clinical epidemiology and health technology development. In these DTA meta‐analyses, some studies may have markedly different characteristics from the others and potentially be inappropriate to include. The inclusion of these “outlying” studies might lead to biases, yielding misleading results. In addition, there might be influential studies that have notable impacts on the results. In this article, we propose Bayesian methods for detecting outlying studies and their influence diagnostics in DTA meta‐analyses. Synthetic influence measures based on the bivariate hierarchical Bayesian random effects models are developed because the overall influences of individual studies should be simultaneously assessed by the two outcome variables and their correlation information. We propose four synthetic measures for influence analyses: (a) relative distance, (b) standardized residual, (c) Bayesian p‐value, and (d) influence statistic on the area under the summary receiver operating characteristic curve. We also show that conventional univariate Bayesian influential measures can be applied to the bivariate random effects models, which can be used as marginal influential measures. Most of these methods can be similarly applied to the frequentist framework. We illustrate the effectiveness of the proposed methods by applying them to a DTA meta‐analysis of ultrasound in screening for vesicoureteral reflux among children with urinary tract infections.

show abstract

Laboratory‐based testing for hepatitis C infection using dried blood spot samples: A systematic review and meta‐analysis of diagnostic accuracy

Carty¹,

McCarthy

O’Neill

et al. 2021

Reviews in Medical Virology

View full text Add to dashboard Cite

The use of dried blood spot (DBS) samples can facilitate the implementation of reflex testing by circumventing the need for centrifugation and freezing of venous blood samples. This systematic review assessed the accuracy of using DBS samples to diagnose chronic hepatitis C virus (HCV) infection. A comprehensive search was undertaken to identify articles published up to July 2020 evaluating the diagnostic accuracy of anti-HCV, HCV-RNA and HCV core antigen tests using DBS. Screening, data extraction, quality appraisal and Grading of Recommendations, Assessment, Development and Evaluations certainty of the evidence assessment were performed independently by two reviewers. Meta-analysis, meta-regression and sensitivity analyses were conducted. The evidence demonstrates that laboratorybased anti-HCV and HCV-RNA tests using DBS samples have high diagnostic accuracy. All comparisons were between DBS and venous samples. For the detection of anti-HCV, sensitivity was 95% (95% CI: 92%-97%) and specificity was 99% ([95% CI: 98%-99%]; n = 25; I 2 = 81%; moderate certainty). For the detection of HCV-RNA, the sensitivity was 95% (95% CI: 93%-97%) and specificity was 97% ([95% CI: 94%-98%]; n = 20; I 2 = 52%; moderate certainty). The sensitivity of HCV core antigen tests was 86% (95% CI: 79%-91%) and specificity was 98% ([95% CI: 94%-99%]; n = 5; I 2 = 37%; low certainty) compared with HCV-RNA (the gold standard for detecting chronic HCV). DBS samples could facilitate diagnosis of chronic HCV infection as the necessary sequential tests (anti-HCV and then HCV-RNA or HCV core antigen) can be undertaken using the same blood sample. This could reduce loss of patient follow-up and support international efforts towards HCV elimination in both high and low prevalence settings.

show abstract

Statistical methods for detecting outlying and influential studies in meta-analysis of diagnostic test accuracy studies

Cited by 11 publications

References 31 publications

Outlier detection and influence diagnostics in network meta‐analysis

Outlier detection and influence diagnostics in network meta‐analysis

Influence diagnostics and outlier detection for meta‐analysis of diagnostic test accuracy

Laboratory‐based testing for hepatitis C infection using dried blood spot samples: A systematic review and meta‐analysis of diagnostic accuracy

Contact Info

Product

Resources

About