Diagnosing diagnostic tests: evaluating the assumptions underlying the estimation of sensitivity and specificity in the absence of a gold standard

Toft, Nils; Jørgensen, Erik; Højsgaard, Søren

doi:10.1016/j.prevetmed.2005.01.006

Cited by 200 publications

(191 citation statements)

References 18 publications

Supporting

Mentioning

186

Contrasting

Unclassified

Order By: Relevance

“…Validity of the statistical results should be assessed by evaluating latent class modeling assumptions (Toft et al, 2005). These assumptions include different disease prevalence among sampled populations, equal diagnostic accuracy across populations, conditional independence between tests, and appropriate specification of prior probabilities.…”

Section: Discussionmentioning

confidence: 99%

Sensitivity and specificity of a hand-held milk electrical conductivity meter compared to the California mastitis test for mastitis in dairy cattle

Fosgate

Petzer

Karzis

2013

The Veterinary Journal

View full text Add to dashboard Cite

Screening tests for mastitis can play an important role in proactive mastitis control programs to reduce the economic impact of this disease. The primary objective of this study was to compare the accuracy of milk electrical resistance (MER) to the California milk cell test (CMCT) in commercial dairy cattle of South Africa using Bayesian methods without a perfect reference test. A total of 1,848 quarter milk specimens were collected from 173 cows sampled during 6 sequential farm visits and 25.8% yielded pathogenic bacterial isolates.Most frequently isolated bacterial species were coagulase negative Staphylococci (n = 346), Streptococcus agalactiae (n = 54), and Staphylococcus aureus (n = 42). The overall cowlevel prevalence of mastitis was 54% based on the Bayesian latent class analysis and the prevalence varied by quarter. The CMCT was more accurate than MER for classification of cows as having somatic cell counts >200,000/ml and isolation of a bacterial pathogen.Bayesian latent class analysis also suggested an overall benefit of CMCT over MER but the difference was not highly statistically probable. The Bayesian model estimated the sensitivity and specificity of MER at a cut-point of >25 mΩ/cm to be 89.9% and 86.8% respectively.The CMCT had a sensitivity and specificity of 94.5% and 77.7% respectively when evaluated at the slight positive cut-point. Milk electrical resistance was useful for identifying milk specimens harboring pathogens but was not able to differentiate among evaluated bacterial isolates. Screening tests can be used to improve udder health as part of a proactive management plan.

show abstract

Section: Discussionmentioning

confidence: 99%

Sensitivity and specificity of a hand-held milk electrical conductivity meter compared to the California mastitis test for mastitis in dairy cattle

Fosgate

Petzer

Karzis

2013

The Veterinary Journal

View full text Add to dashboard Cite

show abstract

“…To determine the test parameters and the true prevalence the HuiWalter paradigm was exploited (Hui and Walter, 1980;Toft et al, 2005). This analysis resulted in an estimated true prevalence of 19% of E. granulosus and 18% for E. multilocularis, respectively.…”

Section: Discussionmentioning

confidence: 99%

“…As two diagnostic procedures were used to determine the infection status in these dogs, this provided a means for estimating the diagnostic parameters if two or more dog populations, which are likely to have different prevalences, are available for investigation using the HuiWalter model for evaluating diagnostic tests in the absence of an absolute standard (Hui and Walter, 1980;Toft et al, 2005). In this case different populations were indicated by the logistic regression.…”

Section: Analysesmentioning

confidence: 99%

“…In addition there is little data on the evaluation of these tests based on samples from endemic areas. Therefore the Hui-Walter model for evaluating diagnostic tests in the absence of any absolute test (Hui and Walter, 1980;Toft et al, 2005) has been used to gain a better insight into the diagnostic performance of these tests in a highly endemic area of Kyrgyzstan.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Canine echinococcosis in Kyrgyzstan: Using prevalence data adjusted for measurement error to develop transmission dynamics models

Ziadinov

Mathis

Trachsel

et al. 2008

International Journal for Parasitology

View full text Add to dashboard Cite

Echinococcosis is a major emerging zoonosis in central Asia. A cross-sectional study of dogs in four villages in rural Kyrgyzstan was undertaken to investigate the epidemiology and transmission of Echinococcus spp. A total of 466 dogs were examined by arecoline purgation for the presence of Echinococcus granulosus and Echinococcus multilocularis. In addition, a faecal sample from each dog was examined for taeniid eggs. Any taeniid eggs found were investigated using PCR techniques (multiplex and single target PCR) to improve the diagnostic sensitivity by confirming the presence of Echinococcus spp. and to identify E. granulosus strains. A total of 83 (18%) dogs had either E. granulosus adults in purge material and/or E. granulosus eggs in their faeces as confirmed by PCR. Three genotypes of E. granulosus: G1, G4 and the G6/7 complex were shown to be present in these dogs through subsequent sequence analysis. Purge analysis combined with PCR identified 50 dogs that were infected with adult E. multilocularis and/or had E. multilocularis eggs in their faeces (11%). Bayesian techniques were employed to estimate the true prevalence, the diagnostic sensitivity and specificity of the procedures used and the transmission parameters. The sensitivity of arecoline purgation for the detection of echinococcosis in dogs was rather low, with a value of 38% (Credible intervals (CIs) 27-50%) for E. granulosus and 21% (CIs 11-34%) for E. multilocularis. The specificity of arecoline purgation was assumed to be 100%. The sensitivity of coproscopy followed by PCR of the isolated eggs was calculated as 78% (CIs 57-87%) for E. granulosus and 50% (CIs 29-72%) for E. multilocularis with specificity of 93% (CIs 88-96%) and 100% (CIs 97-100), respectively. The 93% specificity of the coprological-PCR for E. granulosus could suggest coprophagia rather than true infections. After adjusting for the sensitivity of the diagnostic procedures, the estimated true prevalence of infection of E. granulosus was 19% (CIs 15-25%) and the infection pressure in the dog population was estimated to be 0.29 infections per year (CIs 0.014-0.75). Logistic regression analysis failed to identify any significant risk factors for infections for E. granulosus. After adjusting for the sensitivity of the test procedures, the estimated true prevalence was 18% (CIs 12-30%). Dogs that were restrained had a significantly lower prevalence of E. multilocularis of 11% (CIs 6-29%) compared with 26% in free-roaming dogs (CIs 17-44%) and

show abstract

“…For example, the use of two or more test systems when assessing the prevalence of parasite infection in different groups of animals for transmission studies can be utilised, particularly if there is minimal dependence between the tests and/or one of the test has 100 % specificity (such as PCR or visualisation of the actual parasite or its eggs). The Hui Walter model (Hui & Walter, 1980;Toft et al, 2005) for using multiple diagnostic techniques in the absence of a gold standard can then be incorporated into the transmission model, using for example Bayesian techniques to estimate true prevalences and transmission parameters rather than ones which are based on test prevalences. This is illustrated in a recent manuscript on the transmission of echinococcosis in dogs in Kyrgyzstan (Ziadinov et al, 2008).…”

Section: Diagnostic Uncertaintymentioning

confidence: 99%

Evaluating parasite densities and estimation of parameters in transmission systems

Heinzmann

Torgerson

2008

Parasite

View full text Add to dashboard Cite

Summary :Mathematical modelling of parasite transmission systems can provide useful information about host parasite interactions and biology and parasite population dynamics. In addition good predictive models may assist in designing control programmes to reduce the burden of human and animal disease. Model building is only the first part of the process. These models then need to be confronted with data to obtain parameter estimates and the accuracy of these estimates has to be evaluated. Estimation of parasite densities is central to this. Parasite density estimates can include the proportion of hosts infected with parasites (prevalence) or estimates of the parasite biomass within the host population (abundance or intensity estimates). Parasite density estimation is often complicated by highly aggregated distributions of parasites within the hosts. This causes additional challenges when calculating transmission parameters. Using Echinococcus spp. as a model organism, this manuscript gives a brief overview of the types of descriptors of parasite densities, how to estimate them and on the use of these estimates in a transmission model.

show abstract

Diagnosing diagnostic tests: evaluating the assumptions underlying the estimation of sensitivity and specificity in the absence of a gold standard

Cited by 200 publications

References 18 publications

Sensitivity and specificity of a hand-held milk electrical conductivity meter compared to the California mastitis test for mastitis in dairy cattle

Sensitivity and specificity of a hand-held milk electrical conductivity meter compared to the California mastitis test for mastitis in dairy cattle

Canine echinococcosis in Kyrgyzstan: Using prevalence data adjusted for measurement error to develop transmission dynamics models

Evaluating parasite densities and estimation of parameters in transmission systems

Contact Info

Product

Resources

About