Performance visualization spaces for classification with rejection option

Hanczar, Blaise

doi:10.1016/j.patcog.2019.106984

Cited by 19 publications

(26 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, the second part of Theorem 1 proves that the composite classifier's expected conditional error rate (E[E c D ]) can be expressed in terms of the unconditional error and coverage rates of classifiers A and B, mirroring the relationship in Equation 5 (E c = E u C ). Theorem 2 also proves that this simple formulation for E[E c D ] is equivalent to the non-linear interpolation scheme for conditional error proposed by Hanczar [4]. Note that while their notation interpolates between classifiers X and 0 to produce a classifier 0 + x, we continue to refer to these classifiers as A, B, and D respectively.…”

Section: Proofmentioning

confidence: 67%

“…1d to select their preferred classifier. Alternatively, if the cost of instance rejection (λ r ) can be specified in the range (0, 1) (with 0 denoting the cost of a correct classification and 1 representing the cost of a misclassification), then the cost-optimal classifier can be found by selecting the classifier with minimal expected cost C as defined in Equation 3 (from [4]) and applied in lines 15-17 of Algorithm 1.…”

Section: Proposed Null-labelling Methods For Rejectionmentioning

confidence: 99%

“…All source-code for the experimentation (including dataset pre-processing) has been made available online 4 . The scikitlearn implementation of logistic regression is applied using the lbfgs solver, a maximum of 10,000 iterations (never reached), and a multinomial loss for multi-class datasets.…”

Section: Experimental Studymentioning

confidence: 99%

“…Depending on the use-case, rejected instances may be forwarded to a more resource-heavy classifier or to a human for manual judgement. The field of classification with rejection has also been referred to as learning with abstention [3], selective classification [4], and cautious classification [5].…”

Section: Introductionmentioning

confidence: 99%

“…Fig. 1d provides a comparison between confidencethresholding (CT) and null-labelling (NL) in terms of the tradeoff between the rejection rate and conditional error rate, which is the error rate on non-rejected (covered) instances [4]. As the confidence-threshold is raised (and the rejection rate increases), there is little reduction of the error rate -the rejection method is not much better than randomly selecting instances to reject.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Null-Labelling: A Generic Approach for Learning in the Presence of Class Noise

Denham¹,

Pears²,

Naeem³

2021

Preprint

View full text Add to dashboard Cite

Datasets containing class noise present significant challenges to accurate classification, thus requiring classifiers that can refuse to classify noisy instances. We demonstrate the inability of the popular confidence-thresholding rejection method to learn from relationships between input features and not-at-random class noise. To take advantage of these relationships, we propose a novel null-labelling scheme based on iterative re-training with relabelled datasets that uses a classifier to learn to reject instances that are likely to be misclassified. We demonstrate the ability of null-labelling to achieve a significantly better tradeoff between classification error and coverage than the confidence-thresholding method. Models generated by the null-labelling scheme have the added advantage of interpretability, in that they are able to identify features correlated with class noise. We also unify prior theories for combining and evaluating sets of rejecting classifiers.

show abstract

Section: Proofmentioning

confidence: 67%

Section: Proposed Null-labelling Methods For Rejectionmentioning

confidence: 99%

Section: Experimental Studymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Null-Labelling: A Generic Approach for Learning in the Presence of Class Noise

Denham¹,

Pears²,

Naeem³

2021

Preprint

View full text Add to dashboard Cite

show abstract

Precision and Recall Reject Curves

Fischer,

Wollstadt

2024

Lecture Notes in Networks and Systems

View full text Add to dashboard Cite

Development and evaluation of M + 1-way classification mechanism realized through identifying foreign patterns

Zhang

Zhu²

2022

Soft Comput

View full text Add to dashboard Cite

In this study, we are concerned with a new design methodology of M+1-way classification mechanism. The intent is to reduce the cost of error prediction caused by insufficient evidence. The study is motivated by the notion of three-way decisions, which has been successfully used in various application areas to build human-centric systems. In contrast to traditional multiple classifications, one additional class is added into the proposed architecture to represent the reject decision made on foreign patterns, which exhibit significant differences compared to the patterns used for constructing the classification models. A collection of information granules is constructed on the basis of available experimental evidence to form a compact and interpretable representation of the feature space occupied by the native patterns. The patterns located outside the regions occupied by these information granules are identified and filtered out prior to classification while only the native patterns are subject to classification. The proposed methodology leads to a human-centric and human-interactive construct in which the rejected patterns need further processing. Different distance functions are utilized in the construction of information granules. The performance of the proposed architecture is evaluated involving one synthetic dataset and a collection of publicly available datasets.

show abstract

Performance visualization spaces for classification with rejection option

Cited by 19 publications

References 18 publications

Null-Labelling: A Generic Approach for Learning in the Presence of Class Noise

Null-Labelling: A Generic Approach for Learning in the Presence of Class Noise

Precision and Recall Reject Curves

Development and evaluation of M + 1-way classification mechanism realized through identifying foreign patterns

Contact Info

Product

Resources

About