Consistency thresholds for the planted bisection model

Abstract

E l e c t r o n i c J o u r n a l o f P r o b a b i l i t y Electron. AbstractThe planted bisection model is a random graph model in which the nodes are divided into two equal-sized communities and then edges are added randomly in a way that depends on the community membership. We establish necessary and sufficient conditions for the asymptotic recoverability of the planted bisection in this model. When the bisection is asymptotically recoverable, we give an efficient algorithm that successfully recovers it. We also show that the planted bisection is recoverable asymptotically if and only if with high probability every node belongs to the same community as the majority of its neighbors. Our algorithm for finding the planted bisection runs in time almost linear in the number of edges. It has three stages: spectral clustering to compute an initial guess, a "replica" stage to get almost every vertex correct, and then some simple local moves to finish the job. An independent work by Abbe, Bandeira, and Hall establishes similar (slightly weaker) results but only in the case of logarithmic average degree.Consistency thresholds for the planted bisection model the smallest possible number of edges. This problem is known to be NP-complete in the worst case [14], but on a random graph model with a "planted" small bisection one might hope that it is usually easy. Indeed, Dyer and Frieze showed that if p n = p > q = q n are fixed as n → ∞ then with high probability the bisection that separates the two classes is the minimum bisection, and it can be found in expected O(n 3 ) time.These models were introduced slightly earlier in the statistics literature [12] (under the name "stochastic block model") in order to study the problem of community detection in random graphs. Here, the two parts of the bisection are interpreted as latent "communities" in a network, and the goal is to identify them from the observed graph structure. If p n > q n , the maximum a posteriori estimate of the true communities is exactly the same as the minimum bisection (see the discussion leading to Lemma 4.1), and so the community detection problem on a stochastic block model is exactly the same as the Min-Bisection problem on a planted bisection model; hence, we will use the statistical and computer science terminologies interchangeably. We note, however, the statistics literature is slightly more general, in the sense that it often allows q n > p n , and sometimes relaxes the problem by allowing the detected communities to contain some errors.Our main contribution is a necessary and sufficient condition on p n and q n for recoverability of the planted bisection. When the bisection can be recovered, we provide an efficient algorithm for doing so.

Community detection on euclidean random graphs

“…estimate a corrected community label estimate using the edges not used in the first round. This conjecture is also reminiscent of the 'local to global' phenomena that occurs in many random graph models ( [2], [36], [8], [31]), where an obvious local necessary condition also turns out to be sufficient. However, as a corollary to the GBG algorithm introduced above, we Theorem 17 which establsihes that Exact-Recovery can be solved if the intensity λ is sufficiently high.…”

“…[7], [37]) and computer science (for ex. [30], [12], [36]). The reader should refer to the survey [1] for further background and references on the SBM.…”

Information limits for recovering a hidden community

“…Finally, we briefly compare the results of this paper to those of [1] and [34] on the planted bisection model (also known as the binary symmetric stochastic block model), where the vertices are partitioned into two equal-sized communities. First, a necessary and sufficient condition for weak recovery and a necessary and sufficient condition for exact recovery are obtained in [34].…”

“…First, a necessary and sufficient condition for weak recovery and a necessary and sufficient condition for exact recovery are obtained in [34]. In this paper, sufficient and necessary conditions, (7) and (8) in Theorem 1, are presented separately.…”

“…The idea is to gain independence: the outcome of estimation based on the reduced set of indices is independent of the data between the withheld indices and the reduced set of indices, and the withheld subset is sufficiently small so that we can still obtain sharp constants. This method is mentioned in [14], and variations of it have been used in [14], [35], and [34].…”