Mining the Hidden Link Structure from Distribution Flows for a Spatial Social Network

Abstract

This study aims at developing a non-(semi-)parametric method to extract the hidden network structure from the {0,1}-valued distribution flow data with missing observations on the links between nodes. Such an input data type widely exists in the studies of information propagation process, such as the rumor spreading through social media. In that case, a social network does exist as the media of the spreading process, but its link structure is completely unobservable; therefore, it is important to make inference of the structure (links) of the hidden network. Unlike the previous studies on this topic which only consider abstract networks, we believe that apart from the link structure, different social-economic features and different geographic locations of nodes can also play critical roles in shaping the spreading process, which has to be taken into account. To uncover the hidden link structure and its dependence on the external social-economic features of the node set, a multidimensional spatial social network model is constructed in this study with the spatial dimension large enough to account for all influential social-economic factors. Based on the spatial network, we propose a nonparametric mean-field equation to govern the rumor spreading process and apply the likelihood estimator to make inference of the unknown link structure from the observed rumor distribution flows. Our method turns out easily extendible to cover the class of block networks that are useful in most real applications. The method is tested through simulated data and demonstrated on a data set of rumor spreading on Twitter.

Reconstructing spatial information diffusion networks with heterogeneous agents and text contents

“…We remark that Equation (1) is a counting‐process version of the susceptible–infected–susceptible (SIS) model in Shen et al (2014) and Zhang et al (2018) and of the susceptible–infected (SI) model in Zheng et al (2019). All three models in Shen et al (2014), Zhang et al (2018), and Zheng et al (2019) rely on a binary infection status variable to reconstruct the hidden networks, the difference being that Shen et al (2014) consider only discrete‐time diffusion and apply the compressive sensing algorithm to reconstruct the network, while Zheng et al (2019) deal with continuous‐time diffusion and reconstruct the network via the maximum likelihood procedure.…”

“…We remark that Equation (1) is a counting‐process version of the susceptible–infected–susceptible (SIS) model in Shen et al (2014) and Zhang et al (2018) and of the susceptible–infected (SI) model in Zheng et al (2019). All three models in Shen et al (2014), Zhang et al (2018), and Zheng et al (2019) rely on a binary infection status variable to reconstruct the hidden networks, the difference being that Shen et al (2014) consider only discrete‐time diffusion and apply the compressive sensing algorithm to reconstruct the network, while Zheng et al (2019) deal with continuous‐time diffusion and reconstruct the network via the maximum likelihood procedure. Zhang et al (2018) also deal with the diffusion with a binary status variable and discrete time, but both directions of the transition between the two statuses rely on the mean field determined by the network, and the classical expectation‐maximization algorithm combined with the maximum likelihood procedure is applied to reconstruct the hidden network.…”

“…Under the binary restriction, the conditional expectation dr ( x , i , t ) becomes by definition the conditional probability that the node x turns from susceptible to infected and the left‐hand term dr ( x , i , t )/ dt in Equation (1) becomes the classical Cox proportional hazard function. Then the switch probability from susceptible status to infected status in Shen et al (2014) and Zheng et al (2019) can be recovered from dr ( x , i , t )/ dt via the classical relationship between hazard function and event probability, the technical details of which can be found in Cook and Lawless (2007). In sum, the mean‐field model includes the classical SI and SIS model as special cases, it can be used to fit the binary input data as discussed in Shen et al (2014), Zheng et al (2019), and Zhang et al (2018), but conversely, the models in Shen et al (2014), Zheng et al (2019), Zhang et al (2018) cannot be applied to the countable data format discussed in this article.…”

“…For instance, Shen, Wang, Fan, Di, and Lai (2014) converts the estimation of the binary propagation network adjacency matrix to a classical compressive sensing (CS) problem, and applies convex optimization to solve the problem in a parallel way. Zheng, Zhao, Zhang, Ye, and Dai (2019) considers a continuous time information diffusion process with heterogeneous agents and weighted propagation network, the network and the propagation process are linked through the classical mean‐field equation and the maximum likelihood method is applied to estimate the network weights. More recent studies regarding the propagation network reconstruction technique can be found in Ma, Chen, Lai, and Zhang (2018), Mei et al (2017), Zhang, Xu, Bao, and Ma (2018), Li, Wu, Liu, Lu, and Guo (2015), Liu, Mei, Wu, and Lv (2018).…”

“…For instance, when nodes are identified with households, variables, such as family wealth, education level and religious beliefs, can all be useful features as these socioeconomic factors are influential on the behaviors of these households during information spreading, with which, however, existing studies are rarely concerned. Finally, the existing methods focus more often on the binary propagation process, such as the discussion in Shen et al (2014), Zheng et al (2019), Ma et al (2018), Zhang et al (2018). But in the real world, whether or not a single agent is infected is not always explicitly definable during information diffusion (Zhang et al, 2018).…”