Foundations of Feature Selection

Abstract

In order to confront the problem of the high dimensionality of data, feature selection algorithms have become indispensable components of the learning process. Therefore, a correct selection of the features can lead to an improvement of the inductive learner in terms of learning speed, generalization capacity or simplicity of the induced model. A global overview of the feature selection process is given in Section 2.1. Then, Section 2.2 describes the different types of feature selection methods, as well as providing a description of the most popular algorithms for further analysis and explanation in subsequent chapters of the book. Finally, Section 2.3 summarizes this chapter.In the few last years, several datasets with high dimensionality have become publicly available on the Internet. This fact has posed an interesting challenge to the research community, since for the machine learning methods it is difficult to deal with a large number of input features. At present, the dimensionality of the benchmark datasets that can be found in several data repositories has increased to millions, or even more. In fact, in [1], one of the analyses carried out by the authors reveals that seven out of 11 datasets that have appeared since 2007 exhibit dimensionality in the millions of features. To confront the problem of the large number of input features, dimensionality reduction techniques are indispensable and may help to improve learning performance. The term "Big Dimensionality" has been coined to refer to this issue, in contrast to the "Big Data" problem that deals with extremely huge sample size [1].These dimensionality reduction techniques usually come in two flavors, feature selection and feature extraction, and each of them has its own merits [2]. On the one hand, feature extraction techniques achieve dimensionality reduction by combining the original features. In this manner, they are able to generate a set of new features, which is usually more compact and of stronger discriminating power. It is preferable in applications such as image analysis, signal processing, and information retrieval, where model accuracy is more important than model interpretability. On the other hand, feature selection achieves dimensionality reduction by removing the irrelevant and redundant features [3]. It is widely used in data mining applications, such as text mining, genetics analysis, and sensor data processing. Due to the fact that feature selection maintains the original features, it is especially useful for applications where the original features are important for model interpreting and knowledge extraction. This chapter will present the foundations of feature selection. Feature SelectionFeature selection can be defined as the process of detecting the relevant features and discarding the irrelevant and redundant ones with the goal of obtaining a subset of features that describes properly the given problem with a minimum degradation of performance. It has several advantages [3], such as:• Improving the performance of the m...

Remote sensing of submerged aquatic vegetation: an introduction and best practices review

“…Radiation transfer models use parameters such as sensor altitude, ground elevation, visibility, and atmospheric composition to model the interaction of the radiation through the atmosphere and estimate what portion of the recorded radiation was reflected by the target (Silva et al 2008). A variety of RTMs are available for atmospheric correction-such as MODTRAN (Berk et al 1989), LibRadTran (Mayer and Kylling 2005), and 6SV (Vermote et al 1997)-each with its own strengths and limitations. Such…”

“…Band selection, also referred to as feature selection, entails subsetting the available spectral bands to the optimal set for analysis (Bolón-Canedo et al 2015;Fyfe 2003;Mathur et al 2005;Niroumand-Jadidi et al 2019). Importantly, band selection retains the original units and physical meaning of the data as opposed to transformations.…”

“…Bands are selected if they are both relevant and not redundant by maximizing the accuracy of the analysis (e.g. classification) (Bolón-Canedo et al 2015;van der Heijden et al 2004). A band is considered relevant if its inclusion improves the prediction accuracy of the chosen model, i.e.…”

“…if it contains useful discriminatory information about the target. A band is not redundant if it is not completely correlated with another band or set of bands included in the subset (Bolón-Canedo et al 2015). Feature selection can be performed with a variety of algorithms (see van der Heijden et al 2005 for a review).…”

“…PCA). Though this can produce more compact feature sets with stronger discriminating ability, it also produces features that may not be easily interpretable (Bolón-Canedo et al 2015). The PCA and MNF transformations follow the same principle of producing orthogonal component vectors; PCA orders the vectors according to which explains the greatest variance while MNF orders the transformed components according to the signal to noise ratio (SNR) (Keshava and Mustard 2002).…”