Number Needed to Diagnose, Predict, or Misdiagnose: Useful Metrics for Non-Canonical Signs of Cognitive Status?

Larner, A. J.

doi:10.1159/000492783

Cited by 40 publications

(34 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, various “number needed to” metrics were calculated: number needed to diagnose (NND, where NND = 1/Y); number needed to predict (NNP, where NNP = 1/PSI) [8]; and number needed to misdiagnose (NNM, where NNM = 1/(1 – Acc)) [12]. Also calculated were the “likelihood to be diagnosed or misdiagnosed” ratio (LDM, where LDM = NNM/NND or NNM/NNP) [13,14], and the “summary utility index” (SUI, where SUI = (CUI+ + CUI–)) [15]. The multiplicative inverse of the latter—the “number needed for screening utility” (NNSU = 1/SUI) [15]—was compared to the “number needed to screen” (NNS) metric, the multiplicative inverse of the “identification index” (II) [9], both rounded to the next highest integer value since they represent numbers of patients.…”

Section: Methodsmentioning

confidence: 99%

MACE for Diagnosis of Dementia and MCI: Examining Cut-Offs and Predictive Values

Larner

2019

Diagnostics

View full text Add to dashboard Cite

The definition of test cut-offs is a critical determinant of many paired and unitary measures of diagnostic or screening test accuracy, such as sensitivity and specificity, positive and negative predictive values, and correct classification accuracy. Revision of test cut-offs from those defined in index studies is frowned upon as a potential source of bias, seemingly accepting any biases present in the index study, for example related to sample bias. Data from a large pragmatic test accuracy study examining the Mini-Addenbrooke’s Cognitive Examination (MACE) were interrogated to determine optimal test cut-offs for the diagnosis of dementia and mild cognitive impairment (MCI) using either the maximal Youden index or the maximal correct classification accuracy. Receiver operating characteristic (ROC) and precision recall (PR) curves for dementia and MCI were also plotted, and MACE predictive values across a range of disease prevalences were calculated. Optimal cut-offs were found to be a point lower than those defined in the index study. MACE had good metrics for the area under the ROC curve and for the effect size (Cohen’s d) for both dementia and MCI diagnosis, but PR curves suggested the superiority for MCI diagnosis. MACE had high negative predictive value at all prevalences, suggesting that a MACE test score above either cut-off excludes dementia and MCI in any setting.

show abstract

Section: Methodsmentioning

confidence: 99%

MACE for Diagnosis of Dementia and MCI: Examining Cut-Offs and Predictive Values

Larner

2019

Diagnostics

View full text Add to dashboard Cite

show abstract

“…20 Another formulation of LDM, with the denominator based on predictive values, takes account of disease prevalence. 8,9,17…”

Section: Discussionmentioning

confidence: 99%

“…More recently, another metric attempting to denote test limitation has been introduced: the likelihood to be diagnosed or misdiagnosed (LDM). 8,9 LDM is based on “number needed” metrics which are generally deemed to be more intuitive and hence applicable for both clinicians and patients than Sens and Spec. One form of LDM is given by the ratio of the number needed to misdiagnose, 10 which is the inverse of Inacc, to the number needed to diagnose, which is the inverse of Youden index.…”

Section: Introductionmentioning

confidence: 99%

Cognitive screening instruments for dementia: comparing metrics of test limitation

Larner

2020

Preprint

View full text Add to dashboard Cite

Cognitive screening instruments (CSIs) for dementia and mild cognitive impairment are usually characterised in terms of measures of discrimination such as sensitivity, specificity, and likelihood ratios. However, CSIs also have limitations. Several metrics exist which may be used to denote test limitations but they are seldom examined. Data from several pragmatic test accuracy studies of CSIs were interrogated to calculate various measures of limitation, namely: misclassification rate; net harm to net benefit ratio; and the likelihood to be diagnosed or misdiagnosed. Intra- and inter-test performance for measures of discrimination and limitation were compared. The study found that some tests with very high sensitivity but low specificity for dementia fared poorly on measures of limitation, with high misclassification rates, low net harm to net benefit ratios, and low likelihoods to be diagnosed or misdiagnosed; some had likelihoods favouring misdiagnosis over diagnosis. Tests with a better balance of sensitivity and specificity fared better on measures of limitation. When choosing which CSIs to administer, measures of test limitation should be considered as well as measures of test discrimination. Although high test sensitivity may be desirable to avoid false negatives, false positives also have a cost. Identification of tests having high misclassification rate, low net harm to net benefit ratio, and low likelihood to be diagnosed or misdiagnosed, may have implications for their use in clinical practice.

show abstract

“…6 Analogous to LHH for randomised controlled treatment trials, the 'likelihood to be diagnosed or misdiagnosed' (LDM) metric was developed for use in DTA studies. 7 This is calculated as the ratio of the 'number needed to misdiagnose' (NNM) and either the 'number needed to diagnose' (NND) or the 'number needed to predict' (NNP) (see Table 2 for calculation methods). In any DTA study, NNM will be desirably high (ie few patients misdiagnosed) while NND and NNP will be desirably small (ideally = 1, ie everybody New unitary metrics for dementia test accuracy studies…”

Section: Likelihood To Diagnose or Misdiagnosementioning

confidence: 99%

“…The reciprocal (multiplicative inverse) of SUI was suggested as the 'number needed for screening utility' No cognitive impairment 0.09 [7] 0.51 [7] 0.48 (very poor) [7] 2.08 (poor)…”

Section: Likelihood To Diagnose or Misdiagnosementioning

confidence: 99%

New unitary metrics for dementia test accuracy studies

Larner¹

2019

Prog Neurol Psychiatry

View full text Add to dashboard Cite

Diagnostic test accuracy studies generally present results in terms of the sensitivity, specificity, and predictive values of the test being examined. Although these metrics are familiar, they may be difficult to interpret and apply to individual patients. This article briefly presents some new easily calculated unitary metrics and compares them for some commonly used cognitive screening instruments and neurological signs used to assess cognitive status, highlighting the underlying weakness or ‘fragility’ of these tests.

show abstract

Number Needed to Diagnose, Predict, or Misdiagnose: Useful Metrics for Non-Canonical Signs of Cognitive Status?

Cited by 40 publications

References 23 publications

MACE for Diagnosis of Dementia and MCI: Examining Cut-Offs and Predictive Values

MACE for Diagnosis of Dementia and MCI: Examining Cut-Offs and Predictive Values

Cognitive screening instruments for dementia: comparing metrics of test limitation

New unitary metrics for dementia test accuracy studies

Contact Info

Product

Resources

About