A Cheap Feature Selection Approach for the K-Means Algorithm

Abstract

The increase in the number of features that need to be analyzed in a wide variety of areas, such as genome sequencing, computer vision or sensor networks, represents a challenge for the K-means algorithm. In this regard, different dimensionality reduction approaches for the K-means algorithm have been designed recently, leading to algorithms that have proved to generate competitive clusterings. Unfortunately, most of these techniques tend to have fairly high computational costs and/or might not be easy to parallelize. In this work, we propose a fully-parellelizable feature selection technique intended for the K-means algorithm. The proposal is based on a novel feature relevance measure that is closely related to the K-means error of a given clustering. Given a disjoint partition of the features, the technique consists of obtaining a clustering for each subset of features and selecting the m features with the highest relevance measure. The computational cost of this approach is just O(m • max{n • K, log m}) per subset of features. We additionally provide a theoretical analysis on the quality of the obtained solution via our proposal, and empirically analyze its performance with respect to well-known feature selection and feature extraction techniques. Such an analysis shows that our proposal consistently obtains results with lower K-means error than all the considered feature selection techniques: Laplacian scores, maximum variance, multi-cluster feature selection and random selection, while also requiring similar or lower computational times than these approaches. Moreover, when compared to feature extraction techniques, such as Random Projections, the proposed approach also shows a noticeable improvement in both error and computational time.

Feature selection using self-information and entropy-based uncertainty measure for fuzzy neighborhood rough set

“…After calculation, F(4, 12) = 2.4801 when α = 0.1, and then, we can compute that F = 8.5221 for Classifier KNN and F = 10.0225 for Classifier C4.5. It is apparent to obtain that the two values are greater than the critical value F (4,12). This result exhibits that five methods are significantly different.…”

“…Feature selection is an important data preprocess in the fields of granular computing and artificial intelligence [1][2][3][4][5][6]. Its main goal is to reduce redundant features and simplify the complexity of the classification model, thereby improving the generalization ability of classification model [7][8][9][10][11][12].…”

Crack damage monitoring for compressor blades based on acoustic emission with novel feature and hybridized feature selection

“…Laplacian scores ranks the features based on the intrinsic characteristic and distinction from other features. 40 The nearest neighbor graph G can be calculated using the nearest neighbor method, as expressed in equation (3). where G ij is the value of ij node, x i and x j is the sample i and j , m is the number of samples.…”

FAWPA: A FAW Attack Protection Algorithm Based on the Behavior of Blockchain Miners

“…(3) The clustering algorithm based on cluster center selection represented by K-means is easily affected by the initial cluster center selection. It results in unstable clustering results and makes it easy to fall into the optimal local solution [22].…”