The Ouroboros of Psychological Methodology: The Case of Effect Sizes (Mechanical Objectivity vs. Expertise)

Abstract

The reporting and interpretation of effect sizes is often promoted as a panacea for the ramifications of institutionalized statistical rituals associated with the null-hypothesis significance test. Mechanical objectivity—conflating the use of a method with the obtainment of truth—is a useful theoretical tool for understanding the possible failure of effect size reporting ( Porter, 1995 ). This article helps elucidate the ouroboros of psychological methodology. This is the cycle of improved tools to produce trustworthy knowledge, leading to their institutionalization and adoption as forms of thinking, leading to methodologists eventually admonishing researchers for relying too heavily on rituals, finally leading to the production of more new improved quantitative tools that may follow along this circular path. Despite many critiques and warnings, research psychologists’ superficial adoption of effect sizes might preclude expert interpretation much like in the null-hypothesis significance test as widely received. One solution to this situation is bottom-up: promoting a balance of mechanical objectivity and expertise in the teaching of methods and research. This would require the acceptance and encouragement of expert interpretation within psychological science.

Historically recontextualizing Sidman's Tactics: How behavior analysis avoided psychology's methodological Ouroboros

“…Davidson (2018) provided a framework for looking at the fate of methodological practices in psychology, such as the ritualized NHST usage, using the Ouroboros metaphor, the mythical symbol of endless “cycle of destruction and rebirth” (Merriam‐Webster, 2020). The general tenor of the Ouroboros metaphor in Davidson's analysis is that new tools emerge and are improved upon over time until they become perceived as means for producing trustworthy knowledge.…”

“… General Outline of Stages in Davidson's (2018) Methodological Ouroboros Template (A) and the Framework for Historical Elements in Psychology That Map On to the Template (B) Note . Ouroboros.…”

“…Davidson (2018) argued that psychology lives on mechanical objectivity, by which it tries to arrive at truth, in the methods it uses. These methods are widely adopted in standardized forms.…”

“…Thereafter, textbooks like Guildford's Fundamental Statistics in Psychology and Education appeared in 1942, consolidating “Fisher's logic with the accept‐reject dichotomy of Neyman and Pearson” (Woods, 2011, p. 41), and the hybrid NHST has reigned in psychological research ever since. In light of Davidson's (2018) analysis, the hybrid NHST became a standard tool in psychological research and the research rituals that emerged became cemented by post‐World‐War‐II textbooks. The ritualization has drawn a myriad of criticisms (e.g., Branch, 1999, 2014; Harlow et al, 1997; Lambdin, 2012; Morrison & Henkel, 1970; Rozeboom, 1960; Woods, 2011), which variously decried the pervasive adoption of the hybridized NHST and the conventional p ‐value, the “oft‐neglected role of interpretation” (Davidson, 2018, p. 470) of statistics, and the mindless rituals involved.…”

“…In light of Davidson's (2018) analysis, the hybrid NHST became a standard tool in psychological research and the research rituals that emerged became cemented by post‐World‐War‐II textbooks. The ritualization has drawn a myriad of criticisms (e.g., Branch, 1999, 2014; Harlow et al, 1997; Lambdin, 2012; Morrison & Henkel, 1970; Rozeboom, 1960; Woods, 2011), which variously decried the pervasive adoption of the hybridized NHST and the conventional p ‐value, the “oft‐neglected role of interpretation” (Davidson, 2018, p. 470) of statistics, and the mindless rituals involved. Davidson's main point of contention is that the outcome of the cycle of mechanical objectivity has resulted in the recent recommendations for the adoption of effect sizes such as Cohen's d , as “new tools” to adopt as replacement, which would begin, predictably, another methodological Ouroboros that would meet the same fate to ESs as with NHST previously in psychological research.…”