Measuring what matters in healthcare: a practical guide to psychometric principles and instrument development

Abstract

The provision of quality healthcare relies on scales and measures with robust evidence of their psychometric properties. Using measurement instruments with poor reliability, validity, or feasibility, or those that are not appropriate for the target diagnostic group or construct/dimension under consideration, may be unfavorable for patients, unproductive, and hinder empirical advancement. Resources from the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) group can assist in identifying and developing psychometrically sound measures. The COSMIN initiative is the only international, research-based practice taxonomy and methodological guidelines for measurement in healthcare. This manuscript aims to provide an accessible introduction to theories, principles and practices of psychometrics, instrument properties, and scale development, with applied examples from the COSMIN recommendations. It describes why measurement in healthcare is critical to good practice, explains the concepts of the latent variable and hypothetical construct and their importance in healthcare assessments, explores issues of flawed measurement and briefly explains key theories relevant to psychometrics. The paper also outlines a ten-step process to develop and validate a new measurement instrument, with examples drawn from a recently developed visuoperceptual measure for analysis of disordered swallowing to demonstrate key concepts and provides a guide for understanding properties of and terminology related to measurement instruments. This manuscript serves as a resource for healthcare clinicians, educators, and researchers who seek to develop and validate new measurement instruments or improve the properties of existing ones. It highlights the importance of using psychometrically sound measurement instruments to ensure high-quality healthcare assessments.

A Psychometric Evaluation of the Dysphagia Handicap Index Using Rasch Analysis

“…A measure's robust psychometric properties must be demonstrated before implementing it in healthcare or research [11,12]. Two systematic reviews summarising the evidence for the measurement properties of patient self-reported measures developed for people with dysphagia reported poor and incomplete psychometric data [13,14].…”

“…CTT analyses evaluate the performance of a measure as a whole, whereas the IRT framework uses the item as the unit of analysis. Also, contrary to CTT, results in IRT are not bound by the test population [11,19]. Consequently, psychometric studies using CTT analyses may yield different results than studies incorporating IRT principles and, therefore, may lead to different recommendations or guidelines about which measures to implement in clinics or research [10,15,20].…”