Monte Carlo Feature Selection and Interdependency Discovery in Supervised Classification

Dramiński, Michał; Kierczak, Marcin; Koronacki, Jacek; Komorowski, Jan

doi:10.1007/978-3-642-05179-1_17

Cited by 24 publications

(28 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since the random binary features with low or high indices were defined to have an excess of ones or zeroes, respectively, this corresponds to a weak preference for features with a uniform distribution of values. However, no relation between the value range of a feature and its relative importance was observed, consistent with previously reported results [11], although the variation of the RIs increased slightly with the value range (Fig. 2b).…”

Section: Results Of Simulation Studysupporting

confidence: 93%

“…The use of MCFS was originally illustrated by selecting genes with importance for leukemia and lymphoma [3], and it was later used to study e.g. HIV-1 by selecting residues in the amino acid sequence of reverse transcriptase with importance for drug resistance [4,5]. Furthermore, MCFS may be used to rank the features based on their relative importance score.…”

mentioning

confidence: 99%

“…The strategy of the test is to perform a number of permutations of the decision column, and in each permutation save the highest RI observed for any feature. Thereafter, the test compares the RI of each feature in the original data to the 95 % confidence interval of the mean of the best RI scores [5].…”

mentioning

confidence: 99%

See 2 more Smart Citations

Selection of Significant Features Using Monte Carlo Feature Selection

Bornelöv

Komorowski

2015

Studies in Computational Intelligence

Self Cite

View full text Add to dashboard Cite

Feature selection methods identify subsets of features in large datasets. Such methods have become popular in data-intensive areas, and performing feature selection prior to model construction may reduce the computational cost and improve the model quality. Monte Carlo Feature Selection (MCFS) is a feature selection method aimed at finding features to use for classification. Here we suggest a strategy using a z-test to compute the significance of a feature using MCFS. We have used simulated data with both informative and random features, and compared the z-test with a permutation test and a test implemented into the MCFS software. The z-test had a higher agreement with the permutation test compared with the built-in test. Furthermore, it avoided a bias related to the distribution of feature values that may have affected the built-in test. In conclusion, the suggested method has the potential to improve feature selection using MCFS.

show abstract

Section: Results Of Simulation Studysupporting

confidence: 93%

mentioning

confidence: 99%

See 1 more Smart Citation

Selection of Significant Features Using Monte Carlo Feature Selection

Bornelöv

Komorowski

2015

Studies in Computational Intelligence

Self Cite

View full text Add to dashboard Cite

show abstract

“…Another worth-noticing example of a multivariate feature ranker is the Breiman's relevance measure. It expresses the average increase of a classification error resulting from randomization of attributes that were used during construction of trees by the Random Forest algorithm [140,141].…”

Section: Examples Of Similarity Learning Modelsmentioning

confidence: 99%

Algorithms for Similarity Relation Learning from High Dimensional Data

Janusz

2014

Transactions on Rough Sets XVII

View full text Add to dashboard Cite

Abstract. The notion of similarity plays an important role in machine learning and artificial intelligence. It is widely used in tasks related to a supervised classification, clustering, an outlier detection and planning. Moreover, in domains such as information retrieval or case-based reasoning, the concept of similarity is essential as it is used at every phase of the reasoning cycle. The similarity itself, however, is a very complex concept that slips out from formal definitions. A similarity of two objects can be different depending on a considered context. In many practical situations it is difficult even to evaluate the quality of similarity assessments without considering the task for which they were performed. Due to this fact the similarity should be learnt from data, specifically for the task at hand. This paper presents a research on the problem of similarity learning, which is a part of author's PHD dissertation. It describes a similarity model, called Rule-Based Similarity, and shows algorithms for constructing this model from available data. The model utilizes notions from the rough set theory to derive a similarity function that allows to approximate the similarity relation in a given context. It is largely inspired by the idea of Tversky's feature contrast model and it has several analogical properties. In the paper, those theoretical properties are described and discussed. Moreover, the paper presents results of experiments on real-life data sets, in which a quality of the proposed model is thoroughly evaluated and compared with the state-of-the-art algorithms.

show abstract

“…Many different feature selection methods have been proposed, cf. for example [1], [2], [3], and [4].…”

Section: Preliminary Remarksmentioning

confidence: 99%