Impact of regularization on spectral clustering

Abstract

The performance of spectral clustering can be considerably improved via regularization, as demonstrated empirically in Amini et al. [2]. Here, we provide an attempt at quantifying this improvement through theoretical analysis. Under the stochastic block model (SBM), and its extensions, previous results on spectral clustering relied on the minimum degree of the graph being sufficiently large for its good performance. By examining the scenario where the regularization parameter τ is large we show that the minimum degree assumption can potentially be removed. As a special case, for an SBM with two blocks, the results require the maximum degree to be large (grow faster than log n) as opposed to the minimum degree. More importantly, we show the usefulness of regularization in situations where not all nodes belong to well-defined clusters. Our results rely on a 'bias-variance'-like trade-off that arises from understanding the concentration of the sample Laplacian and the eigen gap as a function of the regularization parameter. As a byproduct of our bounds, we propose a data-driven technique DKest (standing for estimated Davis-Kahan bounds) for choosing the regularization parameter. This technique is shown to work well through simulations and on a real data set.

Change detection in noisy dynamic networks: a spectral embedding approach

“…Vertices in sparse and heterogeneous graphs depict entities with different abilities to establish connections. It is difficult to achieve a good representation if we ignore sparseness and degree heterogeneity when obtaining a low dimensional embedding [22]. By employing ideas from spectral graph theory [23], combined with the graph regularization technique introduced in [24], we formulate a strategy to effectively embed sparse and heterogeneous graphs into low dimensional Euclidean spaces.…”

Concentration of Random Graphs and Application to Community Detection

“…Concentration of Laplacians of random graphs has been studied by [47,16,51,33,26]. Just like the adjacency matrix, the Laplacian is known to concentrate in the dense regime d = Ω(log n), and it fails to concentrate in the sparse regime.…”

“…Perhaps the two most common ones are adding a small constant to all the degrees on the diagonal of D [16], and adding a small constant to all the entries of A before computing the Laplacian. Here we focus on the latter regularization, proposed by [5] and analyzed by [33,26]. Choose τ > 0 and add the same number τ /n to all entries of the adjacency matrix A, thereby replacing it with…”

Optimal Laplacian Regularization for Sparse Spectral Community Detection

“…In the literature, the characterization of the parameter τ was never properly addressed and its assignment was left to a heuristic choice. More specifically, both in [8] and [10] the results provided by the authors seem to suggest a large value of τ , but it is observed experimentally that smaller values of τ give better partitions. In the end, the authors in [8] settle on the choice of τ = 1 n 1 T D1, i.e., the average degree.…”

“…Regularization avoids the spreading of the uninformative bulk and enables the recovery of the low rank structure of the matrix, as depicted in Figure 1.C. Among the many contributions proposing different types of regularization [8,9,10,11,12], we focus on the likely most promising one proposed by [8] that recovers communities from the eigenvectors corresponding to the largest eigenvalues of the matrix L sym…”