Validity arguments for patient-reported outcomes: justifying the intended interpretation and use of data

Abstract

Background Contrary to common usage in the health sciences, the term “valid” refers not to the properties of a measurement instrument but to the extent to which data-derived inferences are appropriate, meaningful, and useful for intended decision making. The aim of this study was to determine how validity testing theory (the Standards for Educational and Psychological Testing) and methodology (Kane’s argument-based approach to validation) from education and psychology can be applied to validation practices for patient-reported outcomes that are measured by instruments that assess theoretical constructs in health. Methods The Health Literacy Questionnaire (HLQ) was used as an example of a theory-based self-report assessment for the purposes of this study. Kane’s five inferences (scoring, generalisation, extrapolation, theory-based interpretation, and implications) for theoretical constructs were applied to the general interpretive argument for the HLQ. Existing validity evidence for the HLQ was identified and collated (as per the Standards recommendation) through a literature review and mapped to the five inferences. Evaluation of the evidence was not within the scope of this study. Results The general HLQ interpretive argument was built to demonstrate Kane’s five inferences (and associated warrants and assumptions) for theoretical constructs, and which connect raw data to the intended interpretation and use of the data. The literature review identified 11 HLQ articles from which 57 sources of validity evidence were extracted and mapped to the general interpretive argument. Conclusions Kane’s five inferences and associated warrants and assumptions were demonstrated in relation to the HLQ. However, the process developed in this study is likely to be suitable for validation planning for other measurement instruments. Systematic and transparent validation planning and the generation (or, as in this study, collation) of relevant validity evidence supports developers and users of PRO instruments to determine the extent to which inferences about data are appropriate, meaningful and useful (i.e., valid) for intended decisions about the health and care of individuals, groups and populations.

Validity Testing and Cultural Adaptation of the eHealth Literacy Questionnaire (eHLQ) Among People With Chronic Diseases in Taiwan: Mixed Methods Study

“…For a translated version to be considered a robust questionnaire in this setting, not only are systematic and standardized translation processes required but also a verification process [16,20,21,43,44]. Our translation process provides evidence for a validity argument of the translated version, as recommended by Hawkins et al [19,20,43].…”

Use of the English Health Literacy Questionnaire (HLQ) with Health Science University Students in Nepal: A Validity Testing Study

“…The five sources of evidence are best employed prospectively to gather information about the interpretation and use of data generated by a new questionnaire or of data generated by a questionnaire in a new context [ 55 ]. This study used a questionnaire in a new context.…”

“…Evidence for the development of the content of the HLQ items has been thoroughly investigated in previous studies [ 30 , 55 ]. In this study, item difficulty was investigated as an indicator of the extent to which respondents understood the content of each item.…”

Cultural and linguistic adaption and testing of the Health Literacy Questionnaire (HLQ) among healthy people in Korea

“…Health literacy impacts health and diseases; therefore, improving health literacy is crucial [ 8 ]. Thus, individuals’ health literacy should be first assessed to provide tailored health information and education, and a valid instrument is required to conduct an accurate health literacy assessment [ 9 , 10 ].…”