Towards an alignment of engineering and psychometric approaches to uncertainty in measurement: Consequences for the future

Abstract

Abstract. The International Vocabulary of Measurement (VIM) and the Guide to Uncertainty in Measurement (GUM) shift the terms and concepts of measurement information quality away from an Error Approach toward a model-based Uncertainty Approach. An analogous shift has taken place in psychometrics with the decreasing use of True Score Theory and increasing attention to probabilistic models for unidimensional measurement. These corresponding shifts emerge from shared roots in cognitive processes common across the sciences and they point toward new opportunities for an art and science of living complex adaptive systems. The psychology of model-based reasoning sets the stage for not just a new consensus on measurement and uncertainty, and not just for a new valuation of the scientific status of psychology and the social sciences, but for an appreciation of how to harness the energy of self-organizing processes in ways that harmonize human relationships.

Item response function in antagonistic situations

“…This probability is the main characteristic of a binary test and is called Item Response Function (IRF). Recently, it was shown that similar models are often explored in different fields of industrial statistics and quality engineering 2‐23 …”

“…Thanks to the authors of References 8‐11, a deep link between the metrological and psychometric approaches was achieved, mainly by adding and adjusting anchoring and resolution parameters to the basic logistic IRF. The issue of similarity between measurement approaches in human, social, and natural (eg, physical) domains has been widely considered in References 12‐15 and used in the framework of the so‐called Rasch model (see Section 3.1 below, as well as References 16,17), including specialists in the field of educational measurement 18 . A wide variety of practical testing problems was considered in Reference 19.…”

Quantifying model selection uncertainty via bootstrapping and Akaike weights

“…For example, ambiguous concept definitions, item selection, and measurement validation contribute to uncertainty in consumer research studies (Rigdon & Sarstedt, 2022). Researchers have therefore started calling for bringing an uncertainty-centric research framework to consumer research (Bach et al, 2020;Fisher & Stenner, 2017;Rigdon et al, 2020), emphasizing that such an undertaking requires considerable effort (Rigdon & Sarstedt, 2022). 1 The aim of this paper is to take a step in that direction by describing an approach for quantifying model selection uncertainty that captures variation in parameter estimates across models.…”