Effect of Population Heterogenization on the Reproducibility of Mouse Behavior: A Multi-Laboratory Study

Abstract

In animal experiments, animals, husbandry and test procedures are traditionally standardized to maximize test sensitivity and minimize animal use, assuming that this will also guarantee reproducibility. However, by reducing within-experiment variation, standardization may limit inference to the specific experimental conditions. Indeed, we have recently shown in mice that standardization may generate spurious results in behavioral tests, accounting for poor reproducibility, and that this can be avoided by population heterogenization through systematic variation of experimental conditions. Here, we examined whether a simple form of heterogenization effectively improves reproducibility of test results in a multi-laboratory situation. Each of six laboratories independently ordered 64 female mice of two inbred strains (C57BL/6NCrl, DBA/2NCrl) and examined them for strain differences in five commonly used behavioral tests under two different experimental designs. In the standardized design, experimental conditions were standardized as much as possible in each laboratory, while they were systematically varied with respect to the animals' test age and cage enrichment in the heterogenized design. Although heterogenization tended to improve reproducibility by increasing within-experiment variation relative to between-experiment variation, the effect was too weak to account for the large variation between laboratories. However, our findings confirm the potential of systematic heterogenization for improving reproducibility of animal experiments and highlight the need for effective and practicable heterogenization strategies.

A glass full of optimism: Enrichment effects on cognitive bias in a rat model of depression

“…By minimizing the error variation, blocking improves the precision of an analysis, thereby avoiding the risk of false positive or negative results. At the same time, using a heterogeneous group of animals increases the external validity of the test outcome and improves reproducibility across experiments Richter, Garner, Auer, Kunert, & Würbel, 2010;Richter, Garner, & Würbel, 2009;Richter et al, 2011).…”

“Personality” in laboratory mice used for biomedical research: A way of understanding variability?

“…Thus, despite rigorous genetic and environmental standardization efforts behavioral experiments may still result in considerable variation if repeated in different (or even the same) laboratories (Crabbe, Wahlsten, & Dudek, 1999;Lewejohann et al, 2006). Noteworthy, exaggerated standardization also comes at the cost of decreasing the external validity and possibly the reproducibility of experimental results (Richter, Garner, & Würbel, 2009;Richter et al, 2011;Würbel, 2000). Thus, suppressing individuality by exaggerated standardization would also imply a dramatic decrease in external validity.…”

Predicting Gene Function from Uncontrolled Expression Variation among Individual Wild-Type Arabidopsis Plants

“…For instance, molecular biologists often profile the mRNA expression response to controlled perturbations, such as environmental or chemical treatments or genetic knockouts. Because reproducibility is a cornerstone of the scientific method, such experiments are invariably performed in a tightly controlled setup (Richter et al, 2011). Great care is taken to control the boundary conditions and to keep unwanted external influences in check.…”