A simulation assessment of methods to infer cultural transmission on dark networks

Abstract

The social transmission of beliefs, behaviors, and technologies is a central function of dark networks, just as it is in legitimate networks. One motivation for disrupting dark networks is to break the flow of information and learning. It is often unclear, however, which network should be targeted for disruption because individuals inhabit multiple and correlated networks, and the most relevant network for a given cultural process must be inferred from limited empirical data. Three analytic methods potentially are able to distinguish among alternative network diffusion processes: autoregression, dyadic regression with permutations, and dyadic regression with or random effects. All three rely on having measureable cultural outcomes and network or tree-like connections among the data points. We tested the ability of each method to infer cultural diffusion correctly within 4000 simulated datasets generated on two historical networks that linked violent and pacifist Anabaptist religious groups. Under both frequentist and Bayesian inference procedures, regression of dyadic matrices with random effects exhibited the best statistical performance. We found similar results in a more comprehensive search of the network parameter space that simulated both network structures and the diffusion of traits.

A Manual for Cultural Analysis

“…In these simulations, we confirmed that linear regression without accounting for nonindependence performed very poorly and with substantially elevated false positive rates. Across all simulated conditions, the random effects method showed the best performance in terms of statistical power and had statistically acceptable false positive rates under conditions in which both lnam and permutations exhibited elevated false positives (Karimov and Matthews, 2017).…”

“…The difficulty, however, with autoregression models for network data is that actual implementations of them have not fully resolved statistical errors that include biased estimates for effect sizes and p-values (Dittrich, Leenders, and Mulder, 2017;Karimov and Matthews, 2017;Matthews et al, 2016;Mizruchi and Neuman, 2008;Neuman and Mizruchi, 2010). In the case of simplified networks that branch, such as phylogenetic trees, the math has been validated numerous times (Felsenstein, 1985;Grafen, 1989;Nunn, 2011;Rohlf, 2006).…”

“…Dyadic regression can be useful particularly when we are trying to evaluate which of several alternative networks is most associated with the cultural measure. There is at least some literature that establishes that these techniques produce acceptable false positive rates in the context of multiple collinear network predictor variables (de Nooy, 2011;Dekker, Krackhardt, and Snijders, 2007;Karimov and Matthews, 2017;O'Malley and Christakis, 2011). The dyadic regression creates pairwise distances from the dependent variable.…”

“…We recently simulated data sets of 45 binary characteristics evolving on the Anabaptist congregational tree or diffusing along the network of connects between group leaders (Karimov and Matthews, 2017). The nodes of the tree and network were 11 congregations of Anabaptists from the Reformation era ( Figures 4.1 and 4.2).…”

“…When we simulated data on the Anabaptist tree and network, we found that the permutation and random effects approaches performed much better than did autoregression as implemented through lnam function in the R package sna. Karimov and Matthews (2017). We also performed a fuller set of simulated inheritance and diffusion where the networks and trees were simulated as well.…”