Applying Kraemer’s Q (Positive Sign Rate): Some Implications for Diagnostic Test Accuracy Study Results

Abstract

Background/Aims: Sensitivity and specificity (Sens, Spec) are not invariant properties of diagnostic and screening tests, but vary in different patient samples. Kraemer [Evaluating medical tests. Objective and quantitative guidelines. 1992] used the level of test, Q, also known as "positive sign rate" (sum of true and false positives divided by sample size), to calculate quality sensitivity and specificity (QSN, QSP). These scaled indices may be more comparable across different patient samples, but have been little studied hitherto. Methods: The dataset of a pragmatic test accuracy study of the Mini-Addenbrooke's Cognitive Examination (MACE) was re-interrogated to calculate values of QSN and QSP and other paired and unitary test outcome measures based on them, and comparison was made with outcomes previously calculated by standard methods. Results: QSN and QSP values in this cohort (n = 755; overall prevalence of dementia and mild cognitive impairment [MCI] 0.15 and 0.29, respectively) were inferior to Sens and Spec, as were all other outcome measures for MACE for the diagnosis of both dementia and MCI. QSN was relatively preserved, indicating the sensitivity of MACE. Conclusion: Indices of test outcome scaled according to Kraemer's Q, the positive sign rate, are less impressive than outcomes calculated by standard methods. These discrepancies may have implications for test evaluation.

Efficiency Index for Binary Classifiers: Concept, Extension, and Application

“…2.2.1) and since balanced accuracy as (Sens + Spec)/2 is familiar, a different terminology is desirable here, so I suggest "balanced level accuracy" (BLAcc) and "balanced level inaccuracy" (BLInacc), where: As shown in the previous section, Sens and Spec are dependent on the level or bias of the test, Q, determined by the test cut-off. One way to accommodate this dependency is to rescale Sens and Spec according to Q, following the method of Kraemer [6], as QSens and QSpec: QSens = (Sens -Q)/Q' QSpec= (Spec -Q')/Q This permits the calculation of various other "quality" measures [17]. For example, quality accuracy (QAcc) and quality inaccuracy (QInacc) may be calculated: As Sens and Spec are unscaled measures, so is Acc.…”

Efficiency Index for Binary Classifiers: Concept, Extension, and Application

“…This permits the calculation of various other "quality" measures [17]. For example, quality accuracy (QAcc) and quality inaccuracy (QInacc) may be calculated: QAcc = (QSens.P) + (QSpec.P ) QInacc = 1 − QAcc QInacc = (1 − QSens).P + (1 − QSpec).P Quality Efficiency index (QEI) is thus:…”