Community Detection in Sparse Realistic Graphs: Improving the Bethe Hessian

Abstract

This article considers the problem of community detection in sparse dynamical graphs in which the community structure evolves over time. A fast spectral algorithm based on an extension of the Bethe-Hessian matrix is proposed, which benefits from the positive correlation in the class labels and in their temporal evolution and is designed to be applicable to any dynamical graph with a community structure. Under the dynamical degree-corrected stochastic block model, in the case of two classes of equal size, we demonstrate and support with extensive simulations that our proposed algorithm is capable of making non-trivial community reconstruction as soon as theoretically possible, thereby reaching the optimal detectability threshold and provably outperforming competing spectral methods.Preprint. Under review.1 In order to design their dynamical non-backtracking matrix, the average number of connections among nodes in the same and across communities must be known. 2 The algorithm a priori requires that η be known; otherwise, η can be estimated through cross-validation. The number of communities instead, is not required as an input of the algorithm.

Nishimori meets Bethe: a spectral method for node classification in sparse weighted graphs

“…This eigenvalue has a modulus greater than the radius of the bulk: its existence and position are known and have been thoroughly investigated [32,33]. There however exists another real isolated eigenvalue with modulus smaller than the radius of the bulk, the existence and importance of which were first evidenced in [34] in the case of unweighted graphs with a community structure. After [34], a similar phenomenon has also been observed in [35] in the context of phase retrieval, relating the Hessian of the TAP free energy and the Bayes optimal inference temperature.…”

“…There however exists another real isolated eigenvalue with modulus smaller than the radius of the bulk, the existence and importance of which were first evidenced in [34] in the case of unweighted graphs with a community structure. After [34], a similar phenomenon has also been observed in [35] in the context of phase retrieval, relating the Hessian of the TAP free energy and the Bayes optimal inference temperature. This isolated eigenvalue inside the bulk of B received less theoretical attention and it is the main object of our central result.…”

“…In this case, the matrix T , where θ T 1 n = n and 1 n θ T θ := Φ. Then, in agreement with [30] and the conjecture of [34], the eigenvalues of B can be described as follows…”

“…The parametrization β = β N is not the only possible choice of β able to reach this threshold. It was indeed shown, under different settings, in [20,27,46,47] that choosing the temperature β = β SG allows one also to achieve non-trivial clustering as soon as theoretically possible.…”

Optimal Laplacian Regularization for Sparse Spectral Community Detection

“…For this choice of r, if the problem is in the easy (polynomial) detectable regime, then only the k smallest eigenvalues of Hr are negative, while s ↑ k+1 (Hr) ≈ 0. In [23] we refined this approach in a two-class setting, showing that there exists a parametrization -depending on the clustering difficultythat leads to better partitions under a generic degree distribution and, at the same time, provides non-trivial clustering as soon as theoretically possible. In [19] we extended our reasoning for more than two classes and studied the shape of the informative eigenvectors.…”