Dealing with the evaluation of supervised classification algorithms

Santafé, Guzmán; Inza, Iñaki; Lozano, José A.

doi:10.1007/s10462-015-9433-y

Cited by 109 publications

(86 citation statements)

References 93 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In particular, in the machine learning field, we may enumerate several kinds of solutions to resolve the problems mentioned above. We may justify a statistic-based approach [57], and more general approach to build ranking method (a ranking-based approach) [2,40,41,79].…”

Section: A2 the Indicators And Methods To Evaluate And Comparison Ofmentioning

confidence: 99%

The BigGrams: the semi-supervised information extraction system from HTML: an improvement in the wrapper induction

Mirończuk

2017

Knowl Inf Syst

View full text Add to dashboard Cite

The aim of this study is to propose an information extraction system, called BigGrams, which is able to retrieve relevant and structural information (relevant phrases, keywords) from semi-structural web pages, i.e. HTML documents. For this purpose, a novel semi-supervised wrappers induction algorithm has been developed and embedded in the BigGrams system. The wrappers induction algorithm utilizes a formal concept analysis to induce information extraction patterns. Also, in this article, the author (1) presents the impact of the configuration of the information extraction system components on information extraction results and (2) tests the boosting mode of this system. Based on empirical research, the author established that the proposed taxonomy of seeds and the HTML tags level analysis, with appropriate pre-processing, improve information extraction results. Also, the boosting mode works well when certain requirements are met, i.e. when well-diversified input data are ensured.

show abstract

Section: A2 the Indicators And Methods To Evaluate And Comparison Ofmentioning

confidence: 99%

The BigGrams: the semi-supervised information extraction system from HTML: an improvement in the wrapper induction

Mirończuk

2017

Knowl Inf Syst

View full text Add to dashboard Cite

show abstract

“…The MCMC procedure is configured to carry out

b i = 100

samples for burn‐in and

s = 1000

samples for calculating the expected values. All the experiments have been validated using a 10 × 5 fold cross validation (CV) …”

Section: Methodsmentioning

confidence: 99%

Learning from Proportions of Positive and Unlabeled Examples

Hernández-González

Inza

Lozano

2016

Int. J. Intell. Syst.

Self Cite

View full text Add to dashboard Cite

Weakly supervised classification tries to learn from data sets which are not certainly labeled. Many problems, with different natures of partial labeling, fit this description. In this paper, the novel problem of learning from positive‐unlabeled proportions is presented. The provided examples are unlabeled, and the only class information available consists of the proportions of positive and unlabeled examples in different subsets of the training data set. We present a methodology that adapts to the different levels of class uncertainty to learn Bayesian network classifiers using an expectation‐maximization strategy. It has been tested in a variety of artificial scenarios with different class uncertainty, as well as compared with two naive strategies that do not consider all the available class information. Finally, it has also been successfully tested in real data, collected from the embryo selection problem in assisted reproduction.

show abstract

“…VIC takes the cluster indexes as class labels and trains an ensemble of supervised classifiers that are then tested using cross‐validation. The average area under the curve (AUC; Santafe et al, ) is the output of the index. The general idea is that the classification performance improves with the quality of the input partition.…”

Section: Related Workmentioning

confidence: 99%

“…Receiver operating characteristic curves are used to evaluate classifiers with continuous outputs that differentiate between two outcomes; they present how the classifier performs in all the range of possible thresholds (Fawcett & 861–874, ). AUC is less sensitive to cost and class imbalance than other nonbalanced scores like classification error; however, it has the limitation that it treats the cost of misclassification differently for each classification algorithm (Santafe et al, ). Nonetheless, AUC is the most used measure for class imbalance problems, (Loyola‐Gonzáez et al, ) and compared with other measures, such as geometric mean or f‐measure, AUC measures more reliably the capability to correctly predict the label of a new object (Japkowicz & Shah, ; Japkowicz, ).…”

Section: Experimental Setup: a Masquerade Detection Settingmentioning

confidence: 99%

Cluster validation in clustering‐based one‐class classification

Rodríguez¹,

Monroy²,

Medina‐Pérez³

et al. 2019

Expert Systems

View full text Add to dashboard Cite

Reconstruction‐based one‐class classification has shown to be very effective in a number of domains. This approach works by attempting to capture the underlying structure of the normal class, typically, by means of clusters of objects. It has the main disadvantage, however, that one has to indicate the number of clusters in advance, for this yields an efficient way of computing a clustering. In this paper, we introduce a new algorithm, OCKRA++, which achieves a better performance, by enhancing a clustering‐based one‐class ensemble classifier (OCKRA) with a cluster validity index that is used to set the best number of clusters during the classifier's training process. We have thoroughly tested OCKRA++ in a particular domain, namely masquerade detection. For this purpose, we have used the Windows‐Users and ‐Intruder simulation Logs data set repository, which contains 70 different masquerade data sets. We have found that OCKRA++ is currently the algorithm that achieves the best area under the curve, with a significant difference, in masquerade detection using the file system navigation approach.

show abstract

Dealing with the evaluation of supervised classification algorithms

Cited by 109 publications

References 93 publications

The BigGrams: the semi-supervised information extraction system from HTML: an improvement in the wrapper induction

The BigGrams: the semi-supervised information extraction system from HTML: an improvement in the wrapper induction

Learning from Proportions of Positive and Unlabeled Examples

Cluster validation in clustering‐based one‐class classification

Contact Info

Product

Resources

About