Metrology of human-based and other qualitative measurements

Abstract

The metrology of human-based and other qualitative measurements is in its infancy-concepts such as traceability and uncertainty are as yet poorly developed. This paper reviews how a measurement system analysis approach, particularly invoking as performance metric the ability of a probe (such as a human being) acting as a measurement instrument to make a successful decision, can enable a more general metrological treatment of qualitative observations. Measures based on human observations are typically qualitative, not only in sectors, such as health care, services and safety, where the human factor is obvious, but also in customer perception of traditional products of all kinds. A principal challenge is that the usual tools of statistics normally employed for expressing measurement accuracy and uncertainty will probably not work reliably if relations between distances on different portions of scales are not fully known, as is typical of ordinal or other qualitative measurements. A key enabling insight is to connect the treatment of decision risks associated with measurement uncertainty to generalized linear modelling (GLM). Handling qualitative observations in this way unites information theory, the perceptive identification and choice paradigms of psychophysics. The Rasch invariant measure psychometric GLM approach in particular enables a proper treatment of ordinal data; a clear separation of probe and item attribute estimates; simple expressions for instrument sensitivity; etc. Examples include two aspects of the care of breast cancer patients, from diagnosis to rehabilitation. The Rasch approach leads in turn to opportunities of establishing metrological references for quality assurance of qualitative measurements. In psychometrics, one could imagine a certified reference for knowledge challenge, for example, a particular concept in understanding physics or for product quality of a certain health care service. Multivariate methods, such as Principal Component Regression, can also be improved by exploiting the increased resolution of the Rasch approach.

Rationale and Design of a Novel Method to Assess the Usability of Body-Worn Absorbent Incontinence Care Products by Caregivers

“…To our knowledge, this is the first application of a Rasch psychometric invariant measurement analysis of usability measures. 15 Tezza and colleagues 20 previously reported a Rasch-type analysis of Web usability. The Rasch approach 22 has been proposed as a viable alternative to methods described in ISO/TS 20282-2:2013, stating that although satisfaction questionnaires produce ordinal data, parametric statistics are more meaningful when analyzing satisfaction questionnaires.…”

“…The weights were then generated to match the relative importance using a Rasch analysis. 15 Finally, target values were determined for each of the chosen usability metrics.…”

“…A metric such as Time on task was analyzed with respect to the distribution of different times and not only the (crude) score of success versus failure. For scoring satisfaction, success was achieved if and only if the “Rasch parameter” (from a Rasch psychometric invariant measurement analysis of the satisfaction questionnaire 15 ) of the test caregiver was greater or equal to the lower specification limit. The chosen specification limits were L SL,Standing = 1.0 and L SL,Lying = 1.0, respectively, which correspond to a success probability of about 74% for an average user.…”

Data generation methods across the empirical sciences: differences in the study phenomena’s accessibility and the processes of data encoding

“…These issues may have contributed to the fact that metrologists' important concerns about the limitations and challenges of 'human-based data generation' (Berglund et al, 2012;Pendrill, 2014;Pendrill & Petersson, 2016) have hardly been noted by psychological and social scientists. And this although metrologists' views can meaningfully complement and expand on the psychologists' and social scientists' increasing criticism of their own methodological practices that, so far, mostly concern the generation and analysis of quantitative data in their fields (e.g., Bruschi 2017;Buntins et al 2016;Hammersley 2013;Morris et al 2017;Tafreshi et al 2016;Valsiner 2017).…”

IUPAC/CITAC Guide: Evaluation of risks of false decisions in conformity assessment of a multicomponent material or object due to measurement uncertainty (IUPAC Technical Report)

“…The JCGM 106 approach was recently extended for conformity assessment in the presence of a systematic measurement error [24] and for qualitative human-based binary nominal and ordinal properties [25,26]. A multivariate data analysis is described in the EURAMET guide to decision-making and conformity assessment [27] using bivariate examples of the 'post office parcel problem' (limitations of a parcel length and girth) and a healthcare study (skin cream friction and adhesion).…”