Feature selection for clustering - a filter solution

Abstract

Processing applications with a large number of dimensions has been a challenge lo the KDD community. Feature selection. an effective dimensionality reduction technique, is an essential pre-processing method to remove noisy features. In rhe literature there are only a few methods pmposed for feature selection for clustering. And, almost all of rhose methods are 'wrapper' techniques that require a clustering algorithm to evaluate the candidate feature subsets. The wrapper approach is largely unsuitable in real-world applications due to its heavy reliance on clustering algarirhms that require parameters such as number of clusters. and due ro lack of suitable clusrering criteria to evaluate clusrering in different subspaces. I n this paper we propose a %Iter' method that is independent of any clusrering algorithm. The proposed method is based on the observation that data with clusters has v e v different point-to-point distance histogram than that of data without clusters. Using this we propose an entropy measure thar is low ifdata has disrinct clusters and high otherwise. The entropy measure is suitable for selecting the most important subset of features because it is invariant with number of dimensions, and is affected only by the quality of clustering. Extensive performance evaluation over synthetic, benchmark, and real datasets shows its effectiveness.

Feature selection in meta learning framework

“…Hence, many methods have been developed to improve the feature selection process, such as wrapper methods [3,4,5], filter methods [6,7], and methods that use fuzzy rough sets [8]. Unfortunately, there is no dominating feature selection method that works best in all cases [9].…”

Local Feature Selection in Text Clustering

“…Feature selection can be classified as either global or local [1]. The global approach aims to select a single subset of features which are relevant to all derived clusters [5].…”

ARE BETTER FEATURE SELECTION METHODS ACTUALLY BETTER? - Discussion, Reasoning and Examples

“…A typical feature selection process consists of four basic steps: feature subset selection, feature subset evaluation, stopping criterion, and result validation. Based on the selection criterion choice, feature selection methods may roughly be divided into three types: the filter (Yu and Liu, 2003;Dash et al, 2002), the wrapper (Kohavi and John, 1997) and the hybrid (Das, 2001;Sebban and Nock, 2002;Somol et al, 2006). The filter model relies on general characteristics of the data to evaluate and select feature subsets without involving any mining algorithm.…”