Selection of Significant Features Using Monte Carlo Feature Selection

Bornelöv, Susanne; Komorowski, Jan

doi:10.1007/978-3-319-18781-5_2

Cited by 6 publications

(3 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For each feature, its RI in the original ranking (without any permutation) is compared against the corresponding distribution of RIs from the experiments with permuted decision attribute (the z test is used this time). This method has been described and studied in Bornelöv and Komorowski (2016). It gives similar results to the previous one but needs more MCFS runs.…”

Section: Determining the Cutoff Valuementioning

confidence: 94%

rmcfs: An R Package for Monte Carlo Feature Selection and Interdependency Discovery

Dramiński¹,

Koronacki²

2018

J. Stat. Soft.

View full text Add to dashboard Cite

We describe the R package rmcfs that implements an algorithm for ranking features from high dimensional data according to their importance for a given supervised classification task. The ranking is performed prior to addressing the classification task per se. This R package is the new and extended version of the MCFS (Monte Carlo feature selection) algorithm where an early version was published in 2005. The package provides an easy R interface, a set of tools to review results and the new ID (interdependency discovery) component. The algorithm can be used on continuous and/or categorical features (e.g., gene expression and phenotypic data) to produce an objective ranking of features with a statistically well-defined cutoff between informative and non-informative ones. Moreover, the directed ID graph that presents interdependencies between informative features is provided.Keywords: MCFS-ID, feature selection, high-dimensional problems, Java, R, ID graph. rmcfs: Monte Carlo Feature Selection and Interdependency Discovery in R 2003). More recently and within non-filter approaches, a Bayesian technique of automatic relevance determination, the use of support vector machines, and the use of ensembles of classifiers, all these either alone or in combination, have proved promising. For further details see Li, Campbell, and Tipping (2002), Lu, Devos, Suykens, Arús, and Huffel (2007), Chrysostomou, Chen, and Liu (2008) and the literature therein.Moreover, the last developments by the late Leo Breiman deserve special attention. In his random forests (RFs), he proposed to make use of the so-called variable (i.e., feature) importance for feature selection. Determination of the importance of the variable is not necessary for random forest construction, but it is a subroutine performed in parallel to building the forest (cf. Breiman and Cutler 2008). Ranking features by variable importance can thus be considered to be a by-product of building the classifier. At the same time, nothing prevents one from using such variable importances within, say, the embedded approach; cf., e.g., Díaz-Uriarte and De Andres (2006). In any case, feature selection by measuring variable importance in random forests should be seen as a very promising method, albeit under one proviso. Namely, the problem with variable importance as originally defined is that it is biased towards variables with many categories; cf. Strobl, Boulesteix, Zeileis, and Hothorn (2007), Archer and Kimes (2008), Nicodemus, Malley, Strobl, and Ziegler (2010). Accordingly, proper debiasing is needed, in order to obtain true ranking of features; cf. Strobl, Boulesteix, Kneib, Augustin, and Zeileis (2008). And, however sound such debiasing may be, it incurs much additional computational cost. For an excellent and recent survey on RFs, their properties and capabilities, see Ziegler and König (2014).Most recently, much work has been done to: (i) give embedded feature selection procedures, in particular those used within RFs (whether biased or unbiased), a clear statistical meaning;...

show abstract

Section: Determining the Cutoff Valuementioning

confidence: 94%

rmcfs: An R Package for Monte Carlo Feature Selection and Interdependency Discovery

Dramiński¹,

Koronacki²

2018

J. Stat. Soft.

View full text Add to dashboard Cite

show abstract

“…In this study, a range of methods were used. These methods are outlined below; they have been reported on extensively previously [ 30‐32,37,39,41‐59 ] and detailed methods are also included as Appendix S1.…”

Section: Methodsmentioning

confidence: 99%

Development of a Gaussian Process – feature selection model to characterise (poly)dimethylsiloxane (Silastic®) membrane permeation

Sun

Hewitt

Wilkinson

et al. 2020

Journal of Pharmacy and Pharmacology

View full text Add to dashboard Cite

Objectives The current study aims to determine the effect of physicochemical descriptor selection on models of polydimethylsiloxane permeation. Methods A total of 2942 descriptors were calculated for a data set of 77 chemicals. Data were processed to remove redundancy, single values, imbalanced and highly correlated data, yielding 1363 relevant descriptors. For four independent test sets, feature selection methods were applied and modelled via a variety of Machine Learning methods. Key findings Two sets of molecular descriptors which can provide improved predictions, compared to existing models, have been identified. Best permeation predictions were found with Gaussian Process methods. The molecular descriptors describe lipophilicity, partial charge and hydrogen bonding as key determinants of PDMS permeation. Conclusions This study highlights important considerations in the development of relevant models and in the construction and use of the data sets used in such studies, particularly that highly correlated descriptors should be removed from data sets. Predictive models are improved by the methodology adopted in this study, notably the systematic evaluation of descriptors, rather than simply using any and all available descriptors, often based empirically on in vitro experiments. Such findings also have clear relevance to a number of other fields.

show abstract

“…Determination of the importance of variables is not compulsory for the construction of random forests, but it is a subroutine to be made corresponding to the construction of the forest [39,40]. Features' ranking by variable significance can thus be considered a by-product of the classifier [41]. In this approach, we rely heavily on using the classifier, and we shall not use it for the classification work per se.…”

Section: Monte Carlo Feature Selection and Interdependency Discovery mentioning

confidence: 99%

A Novel Framework for Selecting Informative Meteorological Stations Using Monte Carlo Feature Selection (MCFS) Algorithm

Niaz

Almanjahie

Ali

et al. 2020

Advances in Meteorology

View full text Add to dashboard Cite

Spatial distribution of meteorological stations has a significant role in hydrological research. The meteorological data play a significant role in drought monitoring; in this regard, accurate and suitable provision of meteorological stations is becoming crucial to improve and strengthen the skill of drought prediction. In this perspective, the choice of meteorological stations in a specific region has substantial importance for accurate estimation and continuous monitoring of drought hazards at the regional level. However, installation and data mining on a large number of meteorological stations require high cost and resources. Therefore, it is necessary to rank and find dependencies among existing meteorological stations in a particular region for further climatological analysis and reanalysis of databases. In this paper, the Monte Carlo feature selection and interdependency discovery (MCFS-ID) algorithm-based framework is proposed to identify the important meteorological station in a particular region. We applied the proposed framework on 12 meteorological stations situated in varying climatological regions of Punjab (Pakistan). We employed the drought index SPTI on 1-, 3-, 6-, 9-, 12-, 24-, and 48-month time-scale data to find the interdependencies among meteorological stations at various locations. We found that Sialkot has significance regional importance for studying SPTI-3, SPTI-6, and SPTI-48 indices. This regional importance is based on scores of relative importance (RI); for example, the RI values for SPTI-3, SPTI-6, and SPTI-48 indices are 0.1570, 0.1080, and 0.0270, respectively. Furthermore, the Jhelum station has more relative importance (RI = 0.1410 and 0.1030) for SPTI-1 and SPTI-9 indices, while varying concentration behaviour is observed in the remaining time scales.

show abstract

Selection of Significant Features Using Monte Carlo Feature Selection

Cited by 6 publications

References 9 publications

rmcfs: An R Package for Monte Carlo Feature Selection and Interdependency Discovery

rmcfs: An R Package for Monte Carlo Feature Selection and Interdependency Discovery

Development of a Gaussian Process – feature selection model to characterise (poly)dimethylsiloxane (Silastic®) membrane permeation

A Novel Framework for Selecting Informative Meteorological Stations Using Monte Carlo Feature Selection (MCFS) Algorithm

Contact Info

Product

Resources

About