Uptake of newer methodological developments and the deployment of meta-analysis in diagnostic test research: a systematic review

Abstract

BackgroundThe last decade has seen a number of methodological developments in meta-analysis of diagnostic test studies. However, it is unclear whether such developments have permeated the wider research community and on which applications they are being deployed. The objective was to assess the uptake and deployment of the main methodological developments in the meta-analysis of diagnostic tests, and identify the tests and target disorders most commonly evaluated by meta-analysis.MethodsDesign - systematic review. Data Sources - Medline, EMBASE, CINAHL, Cochrane, PsychInfo, Global health, HMIC, and AMED were searched for studies published before 31st December 2008. Selection criteria - studies were included if they satisfied all of the following: evaluated a diagnostic test; measured test performance; searched two or more databases; stated search terms and inclusion criteria; used a statistical method to summarise performance. Data extraction - included the following data items: year; test; reference standard; target disorder; setting; statistical and quality methods.Results236 studies were included. Over the last 5 years the number of meta-analyses published has increased, but the uptake of new statistical methods lags behind. Pooling the sensitivity and specificity and using the SROC remain the preferred methods for analysis in 70% of studies, with the bivariate random effects and HSROC model being used in only 22% and 5% of studies respectively. In contrast, between 2006 and 2008 the QUADAS tool was used in 40% of studies. Broadly, radiological imaging was the most frequent category of tests analysed (36%), with cancer (22%) and infection (21%) being the most common categories of target disorder. Nearly 80% of tests analysed were those normally used in specialist settings.ConclusionAlthough quality assessment in meta-analyses has improved with the introduction of QUADAS, uptake of the newer statistical methods is still lagging behind. Furthermore, the focus of secondary research seems to be in evaluating specialist tests in specialist settings, in contrast to the more routine tests and settings encountered in the majority of clinical practice.

What is the test's accuracy in my practice population? Tailored meta-analysis provides a plausible estimate

“…One of the blights of meta-analysis in general is heterogeneity, and it affects around 70% of meta-analyses of DTA studies [2]. Indeed, in each of the three cases analyzed here, the prediction regions were larger than the confidence regions for both tailored and conventional meta-analyses suggesting heterogeneity (Figs.…”

“…Hence, a single study reports on the average performance for the test for circumstances particular to the study's setting. This clearly has implications for meta-analysis and explains why heterogeneity is frequently observed [2].…”

The Moses–Littenberg meta-analytical method generates systematic differences in test accuracy compared to hierarchical meta-analytical models

“…There is a considerable body of systematic reviews and meta-analyses of diagnostic test accuracy (DTA) in the public domain [1], [2]. Meta-analysis allows more precise estimation of test accuracy, can provide a stronger comparison of the accuracy of different tests compared to a single study, and allows the inevitable variability between studies to be quantified and formally investigated, as and when there is sufficient cause to suspect clinical or methodological variability [3], [4].…”

“…Three-quarters of corresponding authors for 24 of the reviews using these “traditional” approaches believed the approach to be “currently recommended” and 71% “believed that the method yielded precise estimates” [16]. Meta-analyses undertaken before this period are even more likely to have used the mathematically inferior models [1], [2].…”

Are regulation-driven performance criteria still acceptable? – The German point of view

“…Most meta-analyses for diagnostic test studies still pool diagnostic sensitivity and specificity values only [7]. The potential medical value is always indirect by changing the pathway of care such as by triggering or stopping a certain drug treatment, a surgical intervention or an additional diagnostic procedure such as a CT-scan.…”