A novel measure for evaluating classifiers

Wei, Jinmao; Yuan, Xiaojie; Hu, Qinghua; Wang, Shuqin

doi:10.1016/j.eswa.2009.11.040

Cited by 57 publications

(47 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The MCC measure was originally extended to the multi-class problem in [14]. Recently, and following a comparison between MCC and Confusion Entropy [40] reported in [17], MCC was recommended as an optimal tool for practical tasks, since it presents a good trade-off among discriminatory ability, consistency and coherent behavior with varying number of classes, unbalanced datasets and randomization.…”

Section: Performance Assessment Measuresmentioning

confidence: 99%

The influence of alignment-free sequence representations on the semi-supervised classification of class C G protein-coupled receptors

Cruz-Barbosa

Vellido

Giraldo

2014

Med Biol Eng Comput

View full text Add to dashboard Cite

G protein-coupled receptors (GPCRs) are integral cell membrane proteins of relevance for pharmacology. The tertiary structure of the transmembrane domain, a gate to the study of protein functionality, is unknown for almost all members of class C GPCRs, which are the target of the current study. As a result, their investigation must often rely on alignments of their amino acid sequences. Sequence alignment entails the risk of missing relevant information. Various approaches have attempted to circumvent this risk through alignment-free transformations of the sequences on the basis of different amino acid physicochemical properties. In this paper, we use several of these alignment-free methods, as well as a basic amino acid composition representation, to transform the available sequences. Novel semi-supervised statistical machine learning methods are then used to discriminate the different class C GPCRs types from the transformed data. This approach is relevant due to the existence of orphan proteins to which type labels should be assigned in a process of deorphanization or reverse pharmacology. The reported experiments show that the proposed techniques provide accurate classification even in settings of extreme class-label scarcity and that fair accuracy can be achieved even with very simple transformation strategies that ignore the sequence ordering.

show abstract

Section: Performance Assessment Measuresmentioning

confidence: 99%

The influence of alignment-free sequence representations on the semi-supervised classification of class C G protein-coupled receptors

Cruz-Barbosa

Vellido

Giraldo

2014

Med Biol Eng Comput

View full text Add to dashboard Cite

show abstract

“…As one can find in [3], the measure of confusion entropy was shown to be more precise than accuracy, for it exploits the class distribution information of misclassifications of all classes. It was also shown to be more precise than RCI (NMI), for it takes into consideration the accuracy of classifiers, as well.…”

Section: Accmentioning

confidence: 99%

“…It also encompasses the measures of the F-score, the sensitivity-PPA (Positive predictive accuracy) average and the sensitivity-PPA product [6], the AUC defined by one run (AUC b ), which is also called balanced accuracy, Youden's index [7,8], which has linear correspondence with AUC b , the odds ratio or cross-product, the discriminant power [9], which can be computed directly from the odds ratio, the likelihood, Cohen's kappa [10,11], relative classifier information (RCI) [12,13], normalized mutual information (NMI) [6,14], Matthews Correlation Coefficient (MCC) [15], the mean F-measure [16], macro average arithmetic [17], macro average geometric [5], etc. Confusion entropy [3], CEN for short, also belongs to this group. All these measures can be computed based on a confusion matrix.…”

Section: Related Workmentioning

confidence: 99%

“…Confusion entropy is introduced for measuring how samples of different classes are mixed. It can be found in [3]; confusion entropy tends to rank the classifier high if samples are unevenly classified into different classes. This implies confusion entropy measures if samples of different classes are well separated in a way different from that of the measures based on ranking.…”

Section: Accmentioning

confidence: 99%

“…In [3], the measure of confusion entropy, CEN for short, was introduced for evaluating classifiers in the multi-class case. By exploiting the misclassification information of confusion matrices, the measure takes into consideration both the classification accuracy and class discrimination power of classifiers.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Probabilistic Confusion Entropy for Evaluating Classifiers

Wang

Wei

Han

et al. 2013

Entropy

Self Cite

View full text Add to dashboard Cite

For evaluating the classification model of an information system, a proper measure is usually needed to determine if the model is appropriate for dealing with the specific domain task. Though many performance measures have been proposed, few measures were specially defined for multi-class problems, which tend to be more complicated than two-class problems, especially in addressing the issue of class discrimination power. Confusion entropy was proposed for evaluating classifiers in the multi-class case. Nevertheless, it makes no use of the probabilities of samples classified into different classes. In this paper, we propose to calculate confusion entropy based on a probabilistic confusion matrix. Besides inheriting the merit of measuring if a classifier can classify with high accuracy and class discrimination power, probabilistic confusion entropy also tends to measure if samples are classified into true classes and separated from others with high probabilities. Analysis and experimental comparisons show the feasibility of the simply improved measure and demonstrate that the measure does not stand or fall over the classifiers on different datasets in comparison with the compared measures.

show abstract

In vivo Raman spectroscopy for detection of oral neoplasia: A pilot clinical study

Krishna

Majumder

Chaturvedi

et al. 2013

Journal of Biophotonics

View full text Add to dashboard Cite

We report a pilot study carried out to evaluate the applicability of in vivo Raman spectroscopy for differential diagnosis of malignant and potentially malignant lesions of human oral cavity in a clinical setting. The study involved 28 healthy volunteers and 171 patients having various lesions of oral cavity. The Raman spectra, measured from multiple sites of normal oral mucosa and of lesions belonging to three histopathological categories, viz. oral squamous cell carcinoma (OSCC), oral submucous fibrosis (OSMF) and leukoplakia (OLK), were subjected to a probability based multivariate statistical algorithm capable of direct multi-class classification. With respect to histology as the gold standard, the diagnostic algorithm was found to provide an accuracy of 85%, 89%, 85% and 82% in classifying the oral tissue spectra into the four tissue categories based on leave-one-subject-out cross validation. When employed for binary classification, the algorithm resulted in a sensitivity and specificity of 94% in discriminating normal from the rest of the abnormal spectra of OSCC, OSMF and OLK tissue sites pooled together.

show abstract

A novel measure for evaluating classifiers

Cited by 57 publications

References 22 publications

The influence of alignment-free sequence representations on the semi-supervised classification of class C G protein-coupled receptors

The influence of alignment-free sequence representations on the semi-supervised classification of class C G protein-coupled receptors

Probabilistic Confusion Entropy for Evaluating Classifiers

In vivo Raman spectroscopy for detection of oral neoplasia: A pilot clinical study

Contact Info

Product

Resources

About