Optimal Neighborhood Kernel Clustering with Multiple Kernels

Abstract

Multiple kernel $k$-means (MKKM) aims to improve clustering performance by learning an optimal kernel, which is usually assumed to be a linear combination of a group of pre-specified base kernels. However, we observe that this assumption could: i) cause limited kernel representation capability; and ii) not sufficiently consider the negotiation between the process of learning the optimal kernel and that of clustering, leading to unsatisfying clustering performance. To address these issues, we propose an optimal neighborhood kernel clustering (ONKC) algorithm to enhance the representability of the optimal kernel and strengthen the negotiation between kernel learning and clustering. We theoretically justify this ONKC by revealing its connection with existing MKKM algorithms. Furthermore, this justification shows that existing MKKM algorithms can be viewed as a special case of our approach and indicates the extendability of the proposed ONKC for designing better clustering algorithms. An efficient algorithm with proved convergence is designed to solve the resultant optimization problem. Extensive experiments have been conducted to evaluate the clustering performance of the proposed algorithm. As demonstrated, our algorithm significantly outperforms the state-of-the-art ones in the literature, verifying the effectiveness and advantages of ONKC.

Local Sample-Weighted Multiple Kernel Clustering With Consensus Discriminative Graph

“…For the post-processing of K * , we perform kernel k-means to obtain the clustering partition and labels whose computational complexity is O(n 3 ). Although the computational complexity of our LSWMKC algorithm is the same as the compared models [14]- [16], [19], [24], [36], [40], [48], [51], its clustering performance exhibits significant improvement as reported in Table II.…”

“…Moreover, to encode the emerging data generated from heterogeneous sources or views, multiple kernel clustering (MKC) provides a flexible and expansive framework for combining a set of kernel matrices since different kernels naturally correspond to different views [12]- [18]. Multiple kernel k-means (MKKM) [19] and various variants are further developed and widely employed in many applications [15], [16], [20]- [23]. Most of the kernel-based algorithms follow a common assumption that all the samples are reliable to exploit the intrinsic structures of data, thus such a globally designed manner equally calculates the pairwise similarities of all samples [15]- [17], [20], [21], [24], [25].…”

Fusion Multiple Kernel K-means

“…Specifically, Avg-KKM (baseline) obtains the consensus kernel by uniformly combines base kernels and then performs kernel k-means on it. We also select five classical algorithms, including MKKM (Huang, Chuang, and Chen 2011), LMKKM (Gönen and Margolin 2014), MKKM-MR (Liu et al 2016), LKAM (Li et al 2016) and ONKC (Liu et al 2017). Additionally, we choose four most recent methods, i.e.…”

“…Multiple kernel clustering (MKC) aims to extract complementary information from multiple pre-specified kernels and then categorize the data with close patterns or structures into the same cluster (Zhao, Kwok, and Zhang 2009;Kloft, Rückert, and Bartlett 2010;Kloft et al 2011;Yu et al 2011;Huang, Chuang, and Chen 2011;Gönen and Alpaydın 2011;Zhou et al 2015;Han et al 2016;Wang et al 2017b;Zhou et al 2021a;Liu et al 2021a). Due to the ability to mining inherent non-linear information, MKC has been intensively researched and commonly applied to various applications (Liu et al 2016;Liu et al 2017;Bhadra, Kaski, and Rousu 2017;Liu et al 2019;Zhou et al 2019Zhou et al , 2021b.…”

“…Ren et al improve the clustering performance of MKC by preserving the global and local graph structures (Ren and Sun 2020). Liu et al assumes that an optimal kernel is in the neighborhood of the mixed kernel to reinforce the presentation of the optimal kernel (Liu et al 2017). Also, multiple kernel kmeans (MKKM) is a popular method, which has been intensively studied and widely applied.…”