Eigenvector selection in spectral clustering using Tabu Search

Abstract

Ng. Jordan Weiss (NJW) is one of the most widely used spectral clustering algorithms. For partitioning data into clusters, this method uses the largest eigenvectors of the normalized affinity matrix derived from the data set. However, this set of features is not always the best selection to represent and reveal the structure of the data. In this paper, we aim to propose a quadratic framework to select the most representative eigenvectors. In this way, we define an objective function which includes two factors. In the first part, the interaction of each pair of eigenvectors is considered. In the second part, the ability of each eigenvector to represent the structure of data is considered separately. Then, we use proposed Tabu Search in [1] to solve this mixed-integer quadratic optimization problem. The experimental results show the success of this method to select relevant eigenvectors.

Feature Selection and Its Use in Big Data: Challenges, Methods, and Trends

“…a small number of parameters to be tuned that are easy to implement and independent of the gradient of an optimization objective, more and more studies have been focused on utilizing these heuristic algorithms to deal with feature selection problems. Representative heuristic algorithms include genetic algorithms [68], [81], [97] [98], differential evolutional algorithms [99], [100], simulated annealing [14], particle swarm optimization [101]- [103], tabu search [104]- [106],and Fisher score algorithms [87], etc. These methods can generally achieve a good feature subset with a fast speed, making the study of feature selection incorporated with search strategies a new trend.…”

Research on Spectral Clustering

“…For NJW which is one of widely-used spectral clustering algorithms, [13] proposes a quadratic framework to select the most representation eigenvectors from the set of all the eigenvectors. First, define an objective function which needs to meet two factors: the function can take into consideration the correlation between each pair eigenvectors; the ability of each eigenvector to represent the structure of data is considered separately.…”

A Novel Spectral Clustering based on Local Distribution

“…There are several papers on this topic. Some of them are due to Higham et al [12], Toussi et al [13] and Qiu et al [14]. A very good survey on spectral clustering is due to Luxburg [15].…”