Mutual Information in Community Detection with Covariate Information and Correlated Networks

Abstract

We study the problem of community detection when there is covariate information about the node labels and one observes multiple correlated networks. We provide an asymptotic upper bound on the per-node mutual information as well as a heuristic analysis of a multivariate performance measure called the MMSE matrix. These results show that the combined effects of seemingly very different types of information can be characterized explicitly in terms of formulas involving low-dimensional estimation problems in additive Gaussian noise. Our analysis is supported by numerical simulations.

Information-Theoretic Limits for the Matrix Tensor Product

“…The role of the coupling matrix in the symmetric model (5) was investigated by Reeves et al [24], [25], who provided onesided bounds on the formulas for the mutual information and MMSE. Subsequent work by Mayya and Reeves [26] extended this approach to the multiview version (6). Concurrent to the work in the present paper, Barbier and Reeves [48] provide an exact characterization for a special case of the multiview model (6) that satisfies an additional convexity property.…”

“…Furthermore, if the convexity condition in (46) holds then, by Proposition 14, we obtain the simplified formula in (49). While the simplified version has appeared in previous work [26], [48] the general case in (48) is new.…”

“…The ability to model networks with both assortative and dissassortative requires a coupling matrix with positive and negative values. A further generalization of (5) is the multiview spiked matrix model used to study community detection with multiple correlated networks [26]. This model is given by…”

“…On the technical side, a further contribution of this paper is the development of techniques for the analysis of highdimensional matrix-valued signals. Our formulas leverage functional properties of entropy under Gaussian convolution, using ideas introduced in analysis of multi-layer networks [19] and, more recently, for matrix factorization problems motivated by community detection [24]- [26]. The proof builds upon the adaptive interpolation method [35], [36], which represents an evolution of earlier interpolation techniques in the statistical physics literature [37], [38].…”

Statistical inference of finite-rank tensors

“…Many inference models can be viewed as special cases of (1.1). Indeed, one could argue that essentially any "fully-connected" inference problem will have the form of (1.1) for some suitable choice of A and p. Among them, the models where the limit free energy has been studied include the spiked Wigner model [1,12,2,18,19], the spiked Wishart model [17,4,11,15,7], the stochastic block model (or community detection problem) [12,16,25], the inference of second order matrix tensor products [24], and the inference of higher order vector tensor products [13,2,18]. The model closest to (1.1) is the inference of finite-rank even-order tensor products studied in [14].…”