The standardization–generalization dilemma: a way out

Abstract

Recently, a debate has emerged on the use and necessity of standardization in experimental testing using animal subjects. The difficulties encountered when trying to reconcile standardization and generalization largely underlie this debate. The more specific the testing procedures are, the less one can generalize to more naturalistic situations, including to human clinical populations. If the goal of a study is to generalize to a larger population, there may be a higher risk attached to false-positive than false-negative results; thus the balance sways toward generalization. Heterogenization of housing conditions and of genetic makeup of experimental animals has been suggested as a possible method to increase the generalizability of results. It is important to remain cognizant, however, of situations in which false negatives can be counterproductive or even dangerous, such as when the goal is to elucidate a physiological mechanism, when expected effect sizes are small, in toxicological studies and in drug safety testing. In such cases, experiments based on standardization may provide more useful information. We pose that it is essential that the goal of the specific experiment conducted is clearly defined and that the decision to balance between standardization and generalization must be made based on the specific needs to meet the intended goal. In this light, we discuss a multi-tiered approach to animal experimentation, in which standardization and generalizability are each given precedence during different phases of a project, depending upon the goal of the experiment.

Assessing learning and memory in pigs

“…To date, there is insufficient knowledge to consider pigs as a standard model for biomedical studies of learning and memory. There is always the danger that a number of these studies have yielded idiosyncratic outcomes (van der Staay 2006), and thus it is important to replicate the results of earlier studies, to consolidate the knowledge base (Muma 1993; van der Staay 2006; van der Staay et al 2009, 2010). …”

“…Behavioral tasks for pigs should in general fulfill a number of criteria:healthy, unimpaired animals should be able to acquire/perform the task;the task should allow a detailed analysis of pigs’ behavior, i.e., it should preferentially provide indices for different behavioral domains (cognitive, sensory, motor, or motivational components) (Wainwright and Colombo 2006);the task should be as stress-free as possible (for both the experimental animal and the experimenter; except if measuring the effects of stress is an explicit aim of the experimental procedure);the task should preferentially tap ecologically relevant behaviors (e.g., to prevent mismatches between the task and the adaptive mechanisms and available behavioral repertoire of the species) (Koolhaas et al 2006);the task should be standardized in order to enable comparisons between studies within and across laboratories (van der Staay et al 2010);the task should, wherever possible, be automated in order to eliminate variability between observers, and to allow fine-tuned analyses;the task should allow investigation of developmental effects (early ontogeny, aging) and should preferentially be suited for repeated testing (van der Staay 2002) in order to allow longitudinal studies;the task should be complex and sensitive enough to capture subtle differences in cognitive abilities (Friess et al 2007; Hagl et al 2005; Laughlin et al 1999). …”

“…the task should be standardized in order to enable comparisons between studies within and across laboratories (van der Staay et al 2010);…”

The surveillance state of behavioral automation

“…One possible mitigation is to heterogenize methodology at each site; however, the resulting variability is the natural enemy of smaller (genetic or other) modifications where behavioral phenotypes might be more subtle yet very informative [15]. A second mitigation would be to reduce or even remove animal handling altogether.…”

Preclinical Phase III Trials in Translational Stroke Research

“…For example, it is hardly possible to reproduce the wide range of age and weight, genetic heterogeneity, comorbidities, immunological conditions, and preexisting medications typically found in stroke patient populations (‘diversity factors’). 4 Neglecting diversity factors could jeopardize our chances to successfully translate basic research findings into novel stroke treatments, and eventually limit the willingness of the public and industry to support preclinical stroke research. In response, Dirnagl and colleagues have recently proposed the concept of ‘phase III’ preclinical trials.…”