Operationalism and realism in psychometrics

Abstract

Psychometrics is one of the main approaches to social scientific measurement. It is relied upon in drug testing, economic policymaking, recruitment, and other decision‐making contexts. The first aim of this article is to introduce philosophers to key aspects of psychometrics, namely, classical test theory, item response theory, and construct validity. The second aim is to show how a debate on the nature of psychological attributes manifests in psychometrics. In this debate, realists claim that psychometric measures are indicators of independently existing qualities, while operationalists argue that a measure defines its target attribute. The third and final aim is to argue that despite its poor reputation among philosophers, operationalism is a viable approach to psychometrics.

A Flexible Inventory of Survey Items for Environmental Concepts Generated via Special Attention to Content Validity and Item Response Theory

“…The validity of the items developed here depends strongly on qualitative assessments of content validity and the assumption that variations in the latent trait causally produce variations in the measurement (i.e., item responses). Anchoring validity to those conditions is consistent with frameworks for philosophy of science known as realism [85] and moderate forms of operationalism-which emphasizes that the meaning of a concept is intimately tied to the method by which the concept is measured [86].…”

Measuring the menu, not the food: “psychometric” data may instead measure “lingometrics” (and miss its greatest potential)

“…Probably the most common approach to empirical psychology is to establish a theoretical relationship between two or more defined constructs, operationalize the constructs as variables and collect some types of data ( Bagozzi and Edwards, 1998 ; Nunally and Bernstein, 2007 ; Borsboom, 2008 ; Bagozzi, 2011 ; Michell, 2013 ; Vessonen, 2019 ; Uher, 2021b ). The testing of the hypotheses, and hence the theories, hinges in the measurement data fitting the predictions, that in turn belong to the argued theories ( Popper and Miller, 1983 ; Jöreskog, 1993 ).…”

Respectful operationalism

“…Second, as the large amount of ink spilled on operationalism testifies, these issues are too vast to be settled in one article. And third, I have addressed some of the long-standing philosophical objections elsewhere (Vessonen, 2019a). However, to assure the reader that RO does not inherit the issues that plague other forms of operationalism, I now outline brief responses to well-known objections.…”