Calculating effect sizes in animal social network analysis

Franks, Daniel W.; Weiss, Michael N.; Silk, Matthew J.; Perryman, Robert J.Y.; Croft, Darren P.

doi:10.1111/2041-210x.13429

Cited by 32 publications

(42 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent studies found that (datastream and node) permutation tests, which are widely used to test hypotheses in animal social network analyses, can produce high rates of type I error (false positives) and type II error (false negatives) (Franks et al 2020; Puga□Gonzalez et al 2020; Weiss et al 2020; Farine and Carter 2020). In order to ensure that the results provided by our GAI analyses are reliable and account for both social and non-social nuisance effects, we used an approach proposed by Farine and Carter (2020) that uses datastream permutations to control for nuisance effects, then uses node permutations to test for the statistical significance of the effect of interest.…”

Section: Methodsmentioning

confidence: 99%

Kinship does not predict the structure of a shark social network

Mourier

Planes

2019

Preprint

View full text Add to dashboard Cite

17The role of relatedness in animal societies has received important consideration as a process 18 driving social groups. In the marine world, most studies which have investigated this question 19 have focused on marine mammals such as whales and dolphins. For sharks, recent studies 20 have demonstrated preferential associations among individuals from which social 21 communities emerge. Assortment patterns have been found according to phenotypic or 22 behavioural traits but the role of genetic relatedness and family structure in shaping the social 23 structure of adult shark populations has never been investigated. Here, we used a social 24 network analysis crossed with DNA microsatellite genotyping to investigate the role of the 25 genetic relatedness in the social structure of a blacktip reef shark (Carcharhinus 26 melanopterus) population. We found an established social network spatially organized but 27 dynamic through time with different communities merging during the mating periods. Such 28 recombination shows the males being more gregarious. Kinship was not a predictor of 29 associations among sharks both at the dyad or community levels as individuals tended to 30 associate independently and randomly regardless of the genetic relatedness between 31 individuals. The lack of parental care in this species may contribute to the breakdown of 32 family links in the population early in life, thereby preventing the formation of kin-based 33 social networks. 34 35

show abstract

Section: Methodsmentioning

confidence: 99%

Kinship does not predict the structure of a shark social network

Mourier

Planes

2019

Preprint

View full text Add to dashboard Cite

show abstract

“…For each run of the simulation, we calculate P values for the effect of the trait value on degree using (1) node permutation tests with the coefficient value as the test statistic, (2) node permutation tests with the coefficient value as the test statistic while controlling for number of observations (Franks et al . 2020), (3) pre-network permutation tests with the coefficient value as the test statistic, (4) pre-network permutation tests with the t statistic as the test statistic, and (5) double permutation tests with the coefficient as the test statistic.…”

Section: Testing the Robustness Of The Double Permutation Approachmentioning

confidence: 99%

“…In summary, our results suggest that double permutation tests are most useful when sampling biases or other nuisance effects might be an issue, and especially when the impact of such effects are expected but not well understood. This method is an alternative to model-fitting methods, such as fitting generalized additive models that can handle non-linearity in the relationship between sampling intensity and a network metric of interest (Franks et al . 2020).…”

Section: The Double Permutation Approach Is Robust To Type I and Typementioning

confidence: 99%

“…Inference will always benefit from relying less on P values and instead focusing more on effect sizes (Nakagawa & Cuthill 2007). Franks et al . (2020) proposed that the coefficients of models can generate reliable relative effect sizes after controlling for the number of observations.…”

Section: The Challenge Of Calculating Effect Sizesmentioning

confidence: 99%

“…Incorporating a specific non-social nuisance effect, such as the number of observations of each individual (which affects network metrics like degree) explicitly into the model can correct the coefficient values. Doing so helps ensure that the zero value of a test statistic (like a t-value or linear slope) accurately represents the null hypothesis of interest, potentially alleviating the need for pre-network permutation tests (Franks et al . 2020).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Permutation tests for hypothesis testing with animal social network data: problems and potential solutions

Farine

Carter

2020

Preprint

View full text Add to dashboard Cite

Generating insights about a null hypothesis requires not only a good dataset, but also statistical tests that are reliable and actually address the null hypothesis of interest. Recent studies have found that permutation tests, which are widely used to test hypotheses when working with animal social network data, can suffer from high rates of type I error (false positives) and type II error (false negatives). Here, we first outline why pre-network and node permutation tests have elevated type I and II error rates. We then propose a new procedure, the double permutation test, that addresses some of the limitations of existing approaches by combining pre-network and node permutations. We conduct a range of simulations, allowing us to estimate error rates under different scenarios, including errors caused by confounding effects of social or non-social structure in the raw data. We show that double permutation tests avoid elevated type I errors, while remaining sufficiently sensitive to avoid elevated type II errors. By contrast, the existing solutions we tested, including node permutations, pre-network permutations, and regression models with control variables, all exhibit elevated errors under at least one set of simulated conditions. Type I error rates from double permutation remain close to 5% in the same scenarios where type I error rates from pre-network permutation tests exceed 30%. The double permutation test provides a potential solution to issues arising from elevated type I and type II error rates when testing hypotheses with social network data. We also discuss other approaches, including restricted node permutations, testing multiple null hypotheses, and splitting large datasets to generate replicated networks, that can strengthen our ability to make robust inferences. Finally, we highlight ways that uncertainty can be explicitly considered during the analysis using permutation-based or Bayesian methods.

show abstract

Common permutation methods in animal social network analysis do not control for non-independence

Hart

Weiss

Brent

et al. 2022

Behav Ecol Sociobiol

View full text Add to dashboard Cite

The non-independence of social network data is a cause for concern among behavioural ecologists conducting social network analysis. This has led to the adoption of several permutation-based methods for testing common hypotheses. One of the most common types of analysis is nodal regression, where the relationships between node-level network metrics and nodal covariates are analysed using a permutation technique known as node-label permutations. We show that, contrary to accepted wisdom, node-label permutations do not automatically account for the non-independences assumed to exist in network data, because regression-based permutation tests still assume exchangeability of residuals. The same assumption also applies to the quadratic assignment procedure (QAP), a permutation-based method often used for conducting dyadic regression. We highlight that node-label permutations produce the same p-values as equivalent parametric regression models, but that in the presence of non-independence, parametric regression models can also produce accurate effect size estimates. We also note that QAP only controls for a specific type of non-independence between edges that are connected to the same nodes, and that appropriate parametric regression models are also able to account for this type of non-independence. Based on this, we suggest that standard parametric models could be used in the place of permutation-based methods. Moving away from permutation-based methods could have several benefits, including reducing over-reliance on p-values, generating more reliable effect size estimates, and facilitating the adoption of causal inference methods and alternative types of statistical analysis.

show abstract

Calculating effect sizes in animal social network analysis

Cited by 32 publications

References 45 publications

Kinship does not predict the structure of a shark social network

Kinship does not predict the structure of a shark social network

Permutation tests for hypothesis testing with animal social network data: problems and potential solutions

Common permutation methods in animal social network analysis do not control for non-independence

Contact Info

Product

Resources

About