Evolving feature selection

Abstract

High-throughput technologies facilitate the measurement of vast numbers of biological variables, thereby providing enormous amounts of multivariate data with which to model biological processes. 1 In translational genomics, phenotype classification via gene expression promises highly discriminatory molecular-based diagnosis, and regulatory-network modeling offers the potential to develop therapeutic strategies based on genomic decision making using classical engineering disciplines such as control theory. 2 Yet one must recognize the obstacles inherent in dealing with extremely large numbers of interacting variables in a nonlinear, stochastic, and redundant system that reacts aggressively to any attempt to probe it-a living system. In particular, large data sets may have the perverse effect of limiting the amount of scientific information that can be extracted, because the ability to build models with scientific validity is negatively impacted by an increasing ratio between the number of variables and the sample size. Our specific interest is in how this dimensionality problem creates the need for feature selection while making feature-selection algorithms less reliable with small samples.Two well-appreciated issues tend to confound feature selection: redundancy and multivariate prediction. Both of these can be illustrated by taking a naïve approach to feature selection by considering all features in isolation, ranking them on the basis of their individual predictive capabilities, selecting some features with the highest individual performances, and then applying a standard classification rule to these features, the reasoning being that these are the best predictors of the class. Redundancy arises because the top-performing features might be strongly related-say, by the fact that they share a similar regulatory pathway-and using more than one or two of them may provide little added benefit. The issue of multivariate prediction arises because top-performing single features may not be significantly more beneficial when used in combination with other features, whereas features that perform poorly when used alone may provide outstanding classifi-Data preprocessing is an indispensable step in effective data analysis. It prepares data for data mining and machine learning, which aim to turn data into business intelligence or knowledge. Feature selection is a preprocessing technique commonly used on highdimensional data. Feature selection studies how to select a subset or list of attributes or variables that are used to construct models describing data. Its purposes include reducing dimensionality, removing irrelevant and redundant features, reducing the amount of data needed for learning, improving algorithms' predictive accuracy, and increasing the constructed models' comprehensibility.Feature selection is different from feature extraction (for example, principal component analysis, singular-value decomposition, manifold learning, and factor analysis), which creates new (ex-tracted) features that are combinations of th...

Feature selection for clustering using instance-based learning by exploring the nearest and farthest neighbors

“…To demonstrate the effectiveness of the selected feature subsets, the subsets are applied to discover clusters whose qualities are evaluated using relative cluster validations. Because different feature subsets have different underlying numbers of natural clusters [26], we measure the experimental results by determining which method obtains clusters that best maximize the between-cluster scatter and minimize the within-cluster scatter.…”

Combining support vector regression with feature selection for multivariate calibration

“…Feature selection is a hot topic in the machine learning and bioinformatics fields [11,[22][23][24], which are categorized into three models, the filter model, the wrapper model and the embedded model where the filter model is independent of the learning machine and both the embedded model and the wrapper model are depending on the learning machine, but the embedded model has lower computational complexity than the wrapper model does and has been widely studied in recent years especially on support vector machines [22,25,26]. Here in this paper, we will apply the filter feature selection model and the embedded model to improve generalization performance of SVR.…”

Modality of Adaptive Neuro-Fuzzy Classifier for Acoustic Signal-Based Traffic Density State Estimation Employing Linguistic Hedges for Feature Selection

“…Some of the well known techniques are heuristic analysis [20], genetic algorithm [17,18], neural networks [19], support vector machine [20], and fuzzy systems [11]. Broadly FS approaches are categorized in two groups: filters and wrappers [16]. In filter approach, the feature selection is run without any learning algorithm.…”