Common permutation methods in animal social network analysis do not control for non-independence

Abstract

The non-independence of social network data is a cause for concern among behavioural ecologists conducting social network analysis. This has led to the adoption of several permutation-based methods for testing common hypotheses. One of the most common types of analysis is nodal regression, where the relationships between node-level network metrics and nodal covariates are analysed using a permutation technique known as node-label permutations. We show that, contrary to accepted wisdom, node-label permutations do not automatically account for the non-independences assumed to exist in network data, because regression-based permutation tests still assume exchangeability of residuals. The same assumption also applies to the quadratic assignment procedure (QAP), a permutation-based method often used for conducting dyadic regression. We highlight that node-label permutations produce the same p-values as equivalent parametric regression models, but that in the presence of non-independence, parametric regression models can also produce accurate effect size estimates. We also note that QAP only controls for a specific type of non-independence between edges that are connected to the same nodes, and that appropriate parametric regression models are also able to account for this type of non-independence. Based on this, we suggest that standard parametric models could be used in the place of permutation-based methods. Moving away from permutation-based methods could have several benefits, including reducing over-reliance on p-values, generating more reliable effect size estimates, and facilitating the adoption of causal inference methods and alternative types of statistical analysis.

Multi-group analysis of grooming network position in a highly social primate

“…Model summaries were used to determine the parametric coefficients of the fixed effects in the remaining models. We refrained from using node-based permutation tests to test significance of the variables as these have been reported to not perform better than parametric regression in terms of returning false positives and negatives [ 73 ]. In the case of multiple comparisons (e.g., when investigating a significant rearing-by-sex interaction), post-hoc comparisons were done with Tukey-adjustments using the emmeans package [ 74 ].…”

Bullying as an advertisement of social dominance in common waxbills

“…We report results from the parametric regression models in the main text, but we also report methods and results for the permutation-based approach in the electronic supplementary material. We tested the prediction that aggressiveness should be higher towards individuals of similar dominance rank, using a multi-membership model with a Markov chain Monte Carlo (MCMC) approach [52]. The response variable in this model was the rate of aggression per dyad of individuals, computed as the total number of aggressive displacements in the dyad divided by the time (in seconds) the two individuals were simultaneously in the feeding area (i.e.…”

Spatiotemporal-social association predicts immunological similarity in rewilded mice

“…The analysis of spatiotemporal/social associations drawn from observation data is considerably more complex and has been the subject of much discussion (65)(66)(67)(68)(69)(70). We opted to use Bayesian multiple-membership mixed-effects linear regressions, which are recognized as a valid statistical approach when analyzing such relationships (66,68) and for which there are statistical packages capable of handling multiple-member mixed-effects beta regressions (brms, which we use).…”