Construction, visualization and application of neutral zone classifiers

Abstract

When the potential for making accurate classifications with a statistical classifier is limited, a neutral zone classifier can be constructed by adding a no-decision option as a classification outcome. We show how a neutral zone classifier can be constructed from a receiving operating characteristic (ROC) curve. We extend the ROC curve graphic to highlight important performance characteristics of a neutral zone classifier. Additional utility of neutral zone classifiers is illustrated by showing how they can be incorporated into the first stage of a two-stage classification process. At the first stage, a classification is attempted from easily collected or inexpensive features. If the classification falls into the neutral zone, additional relatively more expensive features can be obtained and used to make a definitive classification at the second stage. The methods discussed in the paper are illustrated with an application pertaining to prostate cancer.

Adaptive data-driven selection of sequences of biological and cognitive markers in pre-clinical diagnosis of dementia

“…The approaches for sequential classification with mixed-effects models in resent studies (11,16) relied purely on acquired evidence without considering potential benefits of measurements. In the field of neutral zone prediction multiple approaches were presented in the last years (8)(9)(10)36). Application based on such neutral zone classifiers that enable sequential classification were derived and applied to clinical data.…”

“…Conceptually, sequential algorithms based on neutral zone classifiers (8–10) address the use case in which a definitive decision is taken after considering a sequence of measurement by adding d measurements only if an objective criterion for a confident decision is not met. Different algorithms were presented in recent years and successfully applied to clinical data (8, 9, 11). While some only depend on fixed decision boundaries like the posterior class probabilities (11), other approaches consider expected cost reductions of additional measurements in the decisions (8).…”

How to reduce diagnostic error: ‘neutral zone’ approach

“…Multistage or sequential approaches to diagnostic testing are not new in the diagnostic methodology literature; see, for example, the 2003 paper by Thompson4 for a discussion of relevant issues. The method proposed by Jeske et al 3 is closely related to simple ‘AND/OR’ rules for combining diagnostic tests that have been explored elsewhere,5 6 but formalises the way in which they are constructed, illustrated using an example for predicting prostate cancer recurrence. Based on cross-validation results presented for this example, it appears possible that overfitting to the data may be an issue, and this may require further investigation.…”

“…The paper by Jeske et al 3 also introduces a four-way plot that combines the standard ROC plot, showing how sensitivity and specificity vary together as the threshold for test positivity changes, with the predictive ROC plot, which provides similar information based on positive and negative predictive values. Each single point in the plot gives information about the four performance measures at a glance.…”

“…The recent paper by Jeske et al 3 considers methods for identifying what has been termed a ‘neutral zone’, that is, the scenario in which the prediction of an individual’s diagnostic category is uncertain and cannot be definitively classified as either positive or negative. Their approach is a two-stage modelling process that aims to improve false-positive and false-negative rates, and ultimately to reduce costs by identifying the subset of individuals for whom potentially more expensive laboratory tests are needed.…”