Mapping the royal road to science: Towards a heuristic for interpreting experimental evidence

Abstract

The experiment as the gold standard for making claims of causation is a cliché so habitually repeated, that one might forget that in psychological practice experimental results tend to allow multiple explanations. But how can we identify plausible alternative causal explanations for experimental findings? In psychology, the seminal work by Campbell and colleagues on experimental causal inference (specifically their quadrilateral validity taxonomy) has been a pillar of the discipline. The current paper claims that the incorporation of construct validity theory into canonized validity typologies clouds alternative causal pathways through which experimental results might be explained. This paper submits that, by shifting away from construct validity theory towards causally specified assumptions, our conception and hence evaluation of alternative causal hypotheses can be made more rigorous. A change of perspective from validity as a question about “the correctness of a theoretical label” to one about the “causal relationship between instruments (measures and manipulations) and psychological attributes” clarifies our thinking about experimentation, both when interpreting experimental findings and evaluating validity claims. After describing several considerations about measures and manipulations in a causal language, this paper proposes a simple heuristic—the experimental causal inference square (ECIS). The ECIS can serve as a conceptual tool to intepret experimental evidence and identify alternative explanations. The target audience for the ECIS heuristic is the basic researcher interested in developing or appraising causal explanations of experimenal phenomena.