The EFPA Test-Review Model: When Good Intentions Meet a Methodological Thought Disorder

Abstract

The European Federation of Psychologists’ Associations (EFPA) has issued sets of test standards and guidelines for psychometric test reviews without any attempt to address the critical content of many substantive publications by measurement experts such as Joel Michell. For example, he has argued that the psychometric test-theory which underpins classical and modern IRT psychometrics is “pathological”, with the entire profession of psychometricians suffering from a methodological thought disorder. With the advent of new kinds of assessment now being created by the “Next Generation” of psychologists which no longer conform to the item-based, statistical test theory generated last century, a new framework is set out for constructing evidence-bases suitable for these “Next Generation” of assessments, which avoids the illusory beliefs of equal-interval or quantitatively structured psychological attributes. Finally, with no systematic or substantive refutations of the logic, axioms, and evidence set out by Michell and others; it is concluded psychologists and their professional associations remain in denial. As with the eventual demise of a similar attempt to maintain the status quo of professional beliefs within forensic clinical psychology and psychiatry during the last century, those following certain EFPA guidelines might now find themselves required to justify their professional beliefs in legal rather than academic environments.

Transitivity Violations Undermine Rating Scales in Motivation Research

“…However, the prevailing psychological practice ignores the need to explore the inner nature of psychological attributes and continues imposing a pre-defined structure of the measurement model (such as an equal interval rating scale) on a poorly defined psychological attribute. Repeated critiques remain basically unanswered ( Cliff, 1992 ; Barrett, 2003 , 2018 ; Michell , 2003b , 2008 ; Trendler, 2013 ; Uher, 2021 ).…”

Conjoint measurement undone

“…Hence, if one does not want to abstain from using methods of data analysis which rely on the assumption that the measurement problem has been successfully solved (e.g., structural equation modeling, Bollen, 1989; or multiple regression/correlation analysis, Cohen & Cohen, 1975) the question of measurability is to be categorized as urgent. As has been emphasized by Paul Barrett (2008), the consequence of ignoring the issue is scientific stagnation (see also Barrett, 2018). Apparently, the acuteness of the problem is still underestimated.…”

“…Therefore, I believe that the situation is incomparably worse in psychology than the situation in which Joseph Black found himself when developing the theory of latent heat relying on “(relatively) bad measurements” (Bringmann & Eronen, 2016, p. 27; see also Sherry, 2011). In consequence, if measurement is really the aim, “[w]hat psychologists can learn from the history of physics” (Bringmann & Eronen, 2016, p. 27) is that they should not proceed with business as usual and thus preserve the status quo of quantitative psychology as a pathological science (Barrett, 2018; Michell, 2000, 2008), but they should rather follow Regnault’s lead and strive to obtain as good data as possible with as good apparatus as can be manufactured. Otherwise it is advisable to consider procedures of data analysis which are suited for non-quantitative data (e.g., Grice, Barrett, Schlimgen, & Abramson, 2012).…”

Psychometrics is not measurement: Unraveling a fundamental misconception in quantitative psychology and the complex network of its underlying fallacies.

“…They contribute theoretical justification for the increasing scrutiny that practitioners are placing on psychometric scores, such as on their validity as legal evidence in courts. It can no longer be justified that decisions on the application of the death penalty for offenders rests—even if just in part—on psychometrically determined IQ scores expressed to two-decimal place precision (Barrett, 2018), given that measurement is not even involved at all. It is just a matter of time that psychometric scores will be challenged in courts, like forensic psychologists’ and psychiatrists’ diagnostic practices before (Barrett, 2018; Faust, 2012).…”