Evidence for HARKing in mouse behavioural tests of anxiety

Abstract

Anxiety disorders are among the most common mental health conditions needing new and better treatments. Research and development of anxiolytic drugs using animal models heavily rely on behavioural tests, most of which measure anxiolytic effects by increased exploratory activity in potentially threatening environments. However, interpretation of such tests is ambiguous because reduced exploratory activity can reflect either anxiety or sedation by the compound. Based on a systematic review of the sensitivity of mouse behavioural tests of anxiety to anxiolytic drugs, we analysed outcomes of 206 studies testing the effect of diazepam in either the open-field test or the hole-board test, as outcomes of these tests showed substantial variation in both direction and strength of the drug effect. Surprisingly, we found that both the rationale given for using the test, whether to detect anxiolytic or sedative effects, and the predicted effect of diazepam, anxiolytic or sedative, strongly depended on the reported test results. The most likely explanation for such strong dependency is post-hoc reasoning, also called hypothesizing after the results are known (HARKing). HARKing can invalidate study outcomes and hampers evidence synthesis by inflating effect sizes. It may also lead researchers into blind alleys, put patients in clinical trials at risk, and waste animals, time, and resources for inconclusive research.

Animal welfare research is fascinating, ethical, and useful—but how can it be more rigorous?

“…Perhaps worse, we may draw mistaken conclusions about animals: ones which fail to improve their well-being. As Rosso et al [6] argue in a preprint, "HARKing can invalidate study outcomes and hamper evidence synthesis by inflating effect sizes... lead researchers into blind alleys … and waste animals, time, and resources".…”

“…So, how to ensure an indicator has construct validity? Jake Veasey and I [7] outlined three methods: (1) assessing whether a potential indicator changes alongside self-reported affect in humans (assuming homology between species), (2) assessing whether it changes in [6] This meta-analysis shows the dangers of indicators that can be interpreted in diverse ways, and HARK-ing's seductive pull. Are the indicators you use, and the meanings of increased/decreased values, always clear before starting an experiment?…”