Discriminant content validity: A quantitative methodology for assessing content of theory‐based measures, with illustrative applications

Abstract

The DCV method allowed quantitative assessment of each item and can therefore inform the content validity of the measures assessed. The methods can be applied to assess content validity before or after collecting data to select the appropriate items to measure theoretical constructs. Further, the data reported for each item in Appendix S1 can be used in item or measure selection. Statement of contribution What is already known on this subject? There are agreed methods of assessing and reporting construct validity of measures of theoretical constructs, but not their content validity. Content validity is rarely reported in a systematic and transparent manner. What does this study add? The paper proposes discriminant content validity (DCV), a systematic and transparent method of assessing and reporting whether items assess the intended theoretical construct and only that construct. In three studies, DCV was applied to measures of illness perceptions, control cognitions, and theory of planned behaviour response formats. Appendix S1 gives content validity indices for each item of each questionnaire investigated. Discriminant content validity is ideally applied while the measure is being developed, before using to measure the construct(s), but can also be applied after using a measure.

Behaviour change techniques: the development and evaluation of a taxonomic method for reporting and describing behaviour change interventions (a suite of five studies involving consensus methods, randomised controlled trials and analysis of qualitative data)

“…74,77 BCT × domain pairings that had no confidence rating from individual participants (i.e. BCT was not allocated to that domain by that participant) were scored zero and entered into the mean score for that pairing.…”

“…There was no overlap in participants between the 'bottom-up' and 'top-down' sort tasks. The sample size for the closed-sort task was based on estimates given for content-validation exercises, with 2-24 participants being shown to be sufficient [74][75][76][77] and more than five participants reducing the influence of rater outliers. 78 …”

A science for all reasons: A comment on Ogden (2016)

“…We already have enormous 'variability 'of theories and constructs explaining behaviour and behaviour change and would gain by 'eliminating gaps in theories, reducing redundancy, and increasing parsimony' (Hagger, 2009, p.190). There are at least 83 theories of behaviour change and over 1700 constructs (Michie, West, Campbell, Brown Gainforth, 2015) but how many are distinct (Schüz, Sniehotta, Mallach, Wiedemann & Schwarzer, (2007) and how many can be operationalised separately (Johnston, Dixon, Hart, Glidewell, Schröder, & Pollard, 2014)? Creative shifts may lose what was already 'known'; for example, the shift to cognitive models appears to have lost the evidence associated with behavioural models and surprise is expressed that the removal of financial rewards leads to extinction of a learned health-related behaviour, a phenomenon easily explicable by early twentieth century science (Johnston, 2015).…”

Use of the ICF to investigate impairment, activity limitation and participation restriction in people using ankle-foot orthoses to manage mobility disabilities

“…If there is poor content validity and the measures used in a study do not discriminate among constructs, any identified relationships may be of questionable value [30].…”

“…Items from the RAND-36 Item Short Form Health Survey [43] 24,25,26,28,30). All these impairment outcomes have been shown to measure impairment with discriminant validity [44].…”