Clustering Complex Data with Group-Dependent Feature Selection

Abstract

Abstract. We describe a clustering approach with the emphasis on detecting coherent structures in a complex dataset, and illustrate its effectiveness with computer vision applications. By complex data, we mean that the attribute variations among the data are too extensive such that clustering based on a single feature representation/descriptor is insufficient to faithfully divide the data into meaningful groups. The proposed method thus assumes the data are represented with various feature representations, and aims to uncover the underlying cluster structure. To that end, we associate each cluster with a boosting classifier derived from multiple kernel learning, and apply the cluster-specific classifier to feature selection across various descriptors to best separate data of the cluster from the rest. Specifically, we integrate the multiple, correlative training tasks of the cluster-specific classifiers into the clustering procedure, and cast them as a joint constrained optimization problem. Through the optimization iterations, the cluster structure is gradually revealed by these classifiers, while their discriminant power to capture similar data would be progressively improved owing to better data labeling.

Per-Cluster Ensemble Kernel Learning for Multi-Modal Image Clustering With Group-Dependent Feature Selection

“…To this end, we employ the softmax activation function to give the differentiable surrogate of the formulation in MK-SOM. Compared with our prior work [12], [13], it can be verified that the resulting formulation can be much more efficiently optimized by only applying gradient decent methods.…”

“…It follows that the discriminant features for each cluster can be selected across different descriptors and merged to compose the ensemble kernel. Generalized from (10), the objective function of MK-SOM is defined as follows: (12) The resulting constrained optimization problem of MK-SOM becomes (13) subject to for (14)…”

“…For each cluster , and jointly specify how the ensemble kernel and the cluster boundary are constructed. Hence, cluster-dependent feature selection across heterogeneous descriptors is allowed in the formulation (13). For the sake of clearness, the optimization of (13) is described in Section IV-C.…”

“…With the designed loss functions for must-links (20) and cannot-links (22), the objective function pertaining to pairwise constraints is given as follows: (23) By incorporating (23) into (13), the objective function of the proposed approach becomes (24) subject to for (25)…”