Membership representation for detecting block-diagonal structure in low-rank or sparse subspace clustering

Lee, Minsik; Lee, Jieun; Lee, Hyeogjin; Kwak, Nojun

doi:10.1109/cvpr.2015.7298773

Cited by 22 publications

(8 citation statements)

References 25 publications

(56 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other hard separation approaches for spectral clustering include [53], who try to find a rotation of the eigenvectors that brings them as close as possible to a collection of binary vectors, which represent a feature partition. The work of [41] builds on [53] by proposing rotating the eigenvectors to approximately achieve a collection of nonnonegative vectors, followed by a maximum likelihood assignment. Related to this is [32] who search for nonnegative vectors "nearby" the eigenvectors via a penalty term.…”

Section: Separation Of Spectral Clustering Output By Hard Clusteringmentioning

confidence: 99%

Sparse eigenbasis approximation: Multiple feature extraction across spatiotemporal scales with application to coherent set identification

Froyland

ROCK

Sakellariou

2019

Communications in Nonlinear Science and Numerical Simulation

View full text Add to dashboard Cite

The output of spectral clustering is a collection of eigenvalues and eigenvectors that encode important connectivity information about a graph or a manifold. This connectivity information is often not cleanly represented in the eigenvectors and must be disentangled by some secondary procedure. We propose the use of an approximate sparse basis for the space spanned by the leading eigenvectors as a natural, robust, and efficient means of performing this separation. The use of sparsity yields a natural cutoff in this disentanglement procedure and is particularly useful in practical situations when there is no clear eigengap. In order to select a suitable collection of vectors we develop a new Weyl-inspired eigengap heuristic and heuristics based on the sparse basis vectors. We develop an automated eigenvector separation procedure and illustrate its efficacy on examples from time-dependent dynamics on manifolds. In this context, transfer operator approaches are extensively used to find dynamically disconnected regions of phase space, known as almost-invariant sets or coherent sets. The dominant eigenvectors of transfer operators or related operators, such as the dynamic Laplacian, encode dynamic connectivity information. Our sparse eigenbasis approximation (SEBA) methodology streamlines the final stage of transfer operator methods, namely the extraction of almost-invariant or coherent sets from the eigenvectors. It is particularly useful when used on domains with large numbers of coherent sets, and when the coherent sets do not exhaust the phase space, such as in large geophysical datasets.

show abstract

Section: Separation Of Spectral Clustering Output By Hard Clusteringmentioning

confidence: 99%

Sparse eigenbasis approximation: Multiple feature extraction across spatiotemporal scales with application to coherent set identification

Froyland

ROCK

Sakellariou

2019

Communications in Nonlinear Science and Numerical Simulation

View full text Add to dashboard Cite

show abstract

“…An excessively sparse coefficient matrix leads to unsatisfactory clustering results if the non-zero elements do not contain sufficient connections within each subspace [12]. Therefore, numerous methods have been developed for preserving more correlation information but with less sparsity [9], [35], [36].…”

Section: Related Workmentioning

confidence: 99%

“…We construct a new affinity graph according to W * . Subsequently, the normalized cut method [20] is applied to the graph in a way similar to that in [9], [36], [47] to generate the final segmentation results. The main procedures of FGNSC are summarized in Figure 1 and Algorithm 2.…”

Section: Finding Good Neighbors For Subspace Clusteringmentioning

confidence: 99%

“…Such a coefficient matrix can be interpreted as a "similarity matrix", where each entry reflects the similarity of two samples. Therefore, a block-diagonal structure [9] is expected so that the intra-subspace samples are densely connected in the affinity graph and the inter-subspace samples are disconnected. However, it is hard to achieve both benefits within a single model.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Subspace Clustering via Good Neighbors

Yang

Liang

Wang

et al. 2020

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Finding the informative subspaces of high-dimensional datasets is at the core of numerous applications in computer vision, where spectral-based subspace clustering is arguably the most widely studied method due to its strong empirical performance. Such algorithms first compute the affinity matrix to construct a self-representation for each sample using other samples as a dictionary. Sparsity and connectivity of the self-representation play important roles in effective subspace clustering. However, simultaneous optimization of both factors is difficult due to their conflicting nature, and most existing methods are designed to address only one factor. In this paper, we propose a post-processing technique to optimize both sparsity and connectivity by finding good neighbors. Good neighbors induce key connections among samples within a subspace and not only have large affinity coefficients but are also strongly connected to each other. We reassign the coefficients of the good neighbors and eliminate other entries to generate a new coefficient matrix. We show that the few good neighbors can effectively recover the subspace, and the proposed post-processing step of finding good neighbors can be complementary to most existing subspace clustering algorithms. Experiments on five benchmark datasets show that the proposed algorithm performs favorably against the state-of-the-art methods with negligible additional computation cost.

show abstract

“…A large body of work has been conducted on spectral clustering with focus on different aspects and applications [10], [11], [12], [13]. Generally, existing approaches to improving spectral clustering performance can be categorized into two paradigms: (1) how to construct robust affinity matrix (or graphs) so as to improve the clustering performance by using the standard spectral algorithms [14], [15], [16], [17]; (2) how to improve the clustering result when the way of generating a data affinity matrix is fixed [18], [19], [20]. This paper is related to the second paradigm.…”

Section: A Related Workmentioning

confidence: 99%

Convex Sparse Spectral Clustering: Single-View to Multi-View

Yan

Lin

2016

IEEE Trans. on Image Process.

171

View full text Add to dashboard Cite

Spectral Clustering (SC) is one of the most widely used methods for data clustering. It first finds a low-dimensonal embedding U of data by computing the eigenvectors of the normalized Laplacian matrix, and then performs k-means on U to get the final clustering result. In this work, we observe that, in the ideal case, UU should be block diagonal and thus sparse. Therefore we propose the Sparse Spectral Clustering (SSC) method which extends SC with sparse regularization on UU . To address the computational issue of the nonconvex SSC model, we propose a novel convex relaxation of SSC based on the convex hull of the fixed rank projection matrices. Then the convex SSC model can be efficiently solved by the Alternating Direction Method of Multipliers (ADMM). Furthermore, we propose the Pairwise Sparse Spectral Clustering (PSSC) which extends SSC to boost the clustering performance by using the multi-view information of data. Experimental comparisons with several baselines on real-world datasets testify to the efficacy of our proposed methods.

show abstract

Membership representation for detecting block-diagonal structure in low-rank or sparse subspace clustering

Cited by 22 publications

References 25 publications

Sparse eigenbasis approximation: Multiple feature extraction across spatiotemporal scales with application to coherent set identification

Sparse eigenbasis approximation: Multiple feature extraction across spatiotemporal scales with application to coherent set identification

Subspace Clustering via Good Neighbors

Convex Sparse Spectral Clustering: Single-View to Multi-View

Contact Info

Product

Resources

About