Functions of units, scales and quantitative data: Fundamental differences in numerical traceability between sciences

Abstract

Quantitative data are generated differently. To justify inferences about real-world phenomena and establish secured knowledge bases, however, quantitative data generation must follow transparent principles applied consistently across sciences. Metrological frameworks of physical measurement build on two methodological principles that establish transparent, traceable—thus reproducible processes for assigning numerical values to measurands. Data generation traceability requires implementation of unbroken, documented measurand-result connections to justify attributing results to research objects. Numerical traceability requires documented connections of the assigned values to known quantitative standards to establish the results' public interpretability. This article focuses on numerical traceability. It explores how physical measurement units and scales are defined to establish an internationally shared understanding of physical quantities. The underlying principles are applied to scrutinise psychological and social-science practices of quantification. Analyses highlight heterogeneous notions of ‘units’ and ‘scales’ and identify four methodological functions; they serve as (1) ‘instruments’ enabling empirical interactions with study phenomena and properties; (2) structural data format; (3) conceptual data format; and (4) conventionally agreed reference quantities. These distinct functions, employed in different research stages, entail different (if any) rationales for assigning numerical values and for establishing their quantitative meaning. The common numerical recoding of scale categories in tests and questionnaires creates scores devoid of quantitative information. Quantitative meaning is created through numeral-number conflation and differential analyses, producing numerical values that lack systematic relations to known quantity standards regarding the study phenomena and properties. The findings highlight new directions for the conceptualisation and generation of quantitative data in psychology and social sciences.

Rating scales institutionalise a network of logical errors and conceptual problems in research practices: A rigorous analysis showing ways to tackle psychology’s crises

“…different ways, also in psychology, such as counts of test responses of defined correctness (e.g., in attention or achievement tests; Uher, 2020bUher, , 2022.…”

“…This entails serious methodological problems because lack of transparency in data generation cannot be remedied by even the most transparent, sophisticated and robust data analysis of any preregistered study. Transparent data generation requires specification of (1) the system of the empirical phenomena studied (the referents; e.g., behavioural acts), (2) the symbolic study system used to encode and analyse information about that empirical system (the signifiers; e.g., variable values on spreadsheet), and (3) determinative assignment relations between these two study systems (their meanings), so that the same symbol always encodes the same information about the empirical phenomena (Figure 6; Uher, 2018aUher, , 2021aUher, , 2022. This idea is basic also to representational theory of measurement, which formalises axiomatic conditions by which empirical relational structures can be mapped to symbolic relational structures (in representation theorems) as well as permissible operations for transforming the latter without breaking their mapping onto the former (in uniqueness theorems; Krantz et al, 1971;Vessonen, 2017).…”

“…processes that enable measurement and for distinguishing these from other processes of evaluation (e.g., opinion making, assessment). These are the principles of data generation traceability and numerical traceability 22 (Uher, 2020b(Uher, , 2022.…”

“…To justify the attribution of results to the measurands, the entire data generation process must be fully transparent and traceable. This requires operational structures (methods), often implemented through measuring instruments, 24 that enable an empirical interaction with the measurand and that establish proportional, thus quantitative relations between the measurand with conventional definitions of their interrelations (BIPM, 2006;Czichos, 2011;Uher, 2022).…”

“…That is, data variables and values are sign systems that are explored in lieu of the actual study phenomena and the analytical results obtained from these data, in turn, are used to make inferences back to these phenomena ( Figure 6 ). Valid inferences from analytical findings presuppose that it is known what information the data variables and values actually represent, and how exactly they represent the phenomena and properties studied—thus, transparency in data generation (§10 Quantificationism; Uher, 2018a , 2021d , c , 2022 ). This may appear trivial and obvious.…”