Erroneous inference based on a lack of preference within one group: Autism, mice, and the social approach task

Abstract

The Social Approach Task is commonly used to identify sociability deficits when modeling liability factors for autism spectrum disorder (ASD) in mice. It was developed to expand upon existing assays to examine distinct aspects of social behavior in rodents and has become a standard component of mouse ASD-relevant phenotyping pipelines. However, there is variability in the statistical analysis and interpretation of results from this task. A common analytical approach is to conduct within-group comparisons only, and then interpret a difference in significance levels as if it were a group difference, without any direct comparison. As an efficient shorthand, we named this approach EWOCs: Erroneous Withingroup Only Comparisons. Here, we examined the prevalence of EWOCs and used simulations to test whether this approach could produce misleading inferences. Our review of Social Approach studies of high-confidence ASD genes revealed 45% of papers sampled used only this analytical approach. Through simulations, we then demonstrate how a lack of significant difference within one group often does not correspond to a significant difference between groups, and show this erroneous interpretation increases the rate of false positives up to 25%. Finally, we define a simple solution: use an index, like a social preference score, with direct statistical comparisons between groups to identify significant differences. We also provide power calculations to guide sample size in future studies. Overall, elimination of EWOCs and adoption of direct comparisons should result in more accurate, reliable, and reproducible data interpretations from the Social Approach Task across ASD liability models.Lay Summary: The Social Approach Task is widely used to assess social behavior in mice and is frequently used in studies modeling autism. However, reviewing published studies showed nearly half do not use correct comparisons to interpret these data. Using simulated and original data, we argue the correct statistical approach is a direct comparison of scores between groups. This simple solution should reduce false positives and improve consistency of results across studies.

Maternal immune activation in mice disrupts proteostasis in the fetal brain

“…A social preference was calculated as the percentage of time spent investigating the social target out of the total exploration time of both objects. 56 Arenas and contents were thoroughly cleaned between testing sessions. Multiple social targets from different home cages were used for testing to prevent potential odorant confounds from target home cages.…”

Cul3 regulates cytoskeleton protein homeostasis and cell migration during a critical window of brain development

“…Analysis was done using the Noldus EthoVison XT (v.11.5). In addition, the social preference/novelty index for each mouse was calculated, as described previously 55 .…”

Tau Reduction Prevents Key Features of Autism in Mouse Models

“…Partial or complete reduction of tau failed to prevent this deficit in Shank3B À/À mice, although there was a trend in that direction (Figure 7G). Using the more rigorous but less sensitive approach to the statistical analysis of the social preference test recommended by (Nygaard et al, 2019), Shank3B À/À mice of all Mapt genotypes did not differ significantly from WT controls in this test (data not shown).…”