Measurement in psychology: A case of ignoramus et ignorabimus? A rejoinder

Abstract

The Millean Quantity Objection—that is, the claim that the measurement of psychological attributes is impossible (Trendler, 2009)—has been countered with partly vigorous opposition (Kyngdon, 2013; Markus & Borsboom, 2012; Saint-Mont, 2012). Kyngdon’s response is of particular interest, since he asserts that measurement may already have been established. If correct, it would definitely invalidate any quantity objection and end the century-long discussion about the measurability of psychological attributes. Therefore the focus of the rejoinder will be on the question of when measurement is reached. First the meaning of measurement is elaborated. On this basis, criteria for the successful establishment of measurement are formulated and it is outlined how these are satisfied in the case of intensive quantities. It is concluded that the evidence presented by Kyngdon is insufficient and inadequate. The approach will also serve as background to discuss objections raised against the Millean Quantity Objection.

Conjoint measurement undone

“…Without the empirical demonstration of invariance, the “measures” for person ability A or item difficulty D , constructed from “raw scores” (i.e., for person ability the count of items on which the person succeeds and for test difficulty the number of items which the person fails to pass) by means of a measurement model (Wright, 1997), cannot really be considered “measurement values.” Actually, statistical significance does not even guarantee that the calculated “measures” satisfy the properties of an ordinal scale. In general, we are licensed to speak of measurement proper only if the property of invariance is firmly established as an empirical fact (for supplementary criteria of measurement see Trendler, 2013).…”

“…In previous publications, I have already questioned the utility of conjoint measurement in psychology (Trendler, 2009, 2013). My main argument is that in psychology we are not even able to satisfy the first condition of quantity which demands the identification of equal magnitudes of quantity.…”

“…For instance, consider cognitive psychology, where psychologists attempt to unravel complex “internal” systems (e.g., the structure of memory) based on a small “external” basis (e.g., reaction time measurements). As I have argued elsewhere (Trendler, 2013), this endeavor is critically underdetermined , i.e., the empirical data are not able to put any meaningful restraint on theory. Cognitive psychologists are not even able to uniquely decide in which format (i.e., digital or analog) information is encoded in the brain.…”

The EFPA Test-Review Model: When Good Intentions Meet a Methodological Thought Disorder

“…Indeed, this “manipulation requirement” is the very issue that Trendler [ 23 , 51 , 52 ] has addressed in his articles, the impossibility of manipulating the cause of any psychological attribute in such a way as to confirm/disconfirm the homomorphic relation to a numerical or order-relation system used to represent magnitudes of an attribute. It was thought this might be achieved by using additive/simultaneous conjoint measurement but Trendler [ 23 ] has shown that this is after all, impossible.…”

On the Philosophical Foundations of Psychological Measurement

“…Finkelstein (e.g., [24,25]) drew a relevant distinction between the measurement of ''hard" and ''soft" systems, describing the latter in terms of domains that involve ''human action, perception, feeling, decisions and the like" [25, p. 269], and noting that invariant relations could likely not be established amongst ''soft" systems due to the absence of ''adequately complete" and validated theories. A variety of sub-fields in the psychological sciences (including psychometrics, econometrics, mathematical psychology, and psychophysics) have developed techniques that purport to permit the measurement of attributes in ''soft" systems, but the claims made in these subfields remain controversial; notably, in recent years, a number of scholars (e.g., [8][9][10]12,13,11,19,28,[34][35][36]41,[43][44][45][46][47]49,50,63,65,67,73,77,78]) have subjected the conceptual and philosophical foundations of psychological measurement to vigorous investigation and critique. The conclusions of these inquiries have often turned up unfavourably regarding both the actuality and even the possibility of psychological measurement (for the former, see in particular [44,45,46,47,49,50]; for the latter, see [77,78]), evidencing that the way in which measurement is understood by psychological scientists may be entirely dissimilar to the way in which it is understood by physical scientists and philosophers of science.…”