Best Practices for Developing and Validating Scales for Health, Social, and Behavioral Research: A Primer

Abstract

Scale development and validation are critical to much of the work in the health, social, and behavioral sciences. However, the constellation of techniques required for scale development and evaluation can be onerous, jargon-filled, unfamiliar, and resource-intensive. Further, it is often not a part of graduate training. Therefore, our goal was to concisely review the process of scale development in as straightforward a manner as possible, both to facilitate the development of new, valid, and reliable scales, and to help improve existing ones. To do this, we have created a primer for best practices for scale development in measuring complex phenomena. This is not a systematic review, but rather the amalgamation of technical literature and lessons learned from our experiences spent creating or adapting a number of scales over the past several decades. We identified three phases that span nine steps. In the first phase, items are generated and the validity of their content is assessed. In the second phase, the scale is constructed. Steps in scale construction include pre-testing the questions, administering the survey, reducing the number of items, and understanding how many factors the scale captures. In the third phase, scale evaluation, the number of dimensions is tested, reliability is tested, and validity is assessed. We have also added examples of best practices to each step. In sum, this primer will equip both scientists and practitioners to understand the ontology and methodology of scale development and validation, thereby facilitating the advancement of our understanding of a range of health, social, and behavioral outcomes.

The development and validation of the muscularity‐oriented eating test: A novel measure of muscularity‐oriented disordered eating

“…Item distributions were assessed to consider the estimation method for EFA and CFA (multivariate distribution; Boateng et al, ; Swami & Barron, in press). Multivariate normality was assessed through the Mardia's test (skewness 611.80, χ 2 (15, 180) = 29,278.26, p < .001; kurtosis χ 2 (1) = 2,484.37, p < .001).…”

“…Factors in the actual data are only retained if their eigenvalues are greater than the eigenvalues from the random data (Hayton, Allen, & Scarpello, ). We removed items with insufficient factor loadings (<0.60; Hair, Black, Babin, & Anderson, ; Swami & Barron, in press) and inter‐item or item‐total correlations <.30 in Study 1 (Boateng et al, ).…”

“…In determining sample size for the exploratory factor analysis (EFA) (Study 1), approximately 300 participants were sought following guidelines for scale development (Boateng, Neilands, Frongillo, Melgar-Quiñonez, & Young, 2018) and findings from a simulation study for development of a single-factor scale (MacCallum, Widaman, Zhang, & Hong, 1999). In determining sample size for the confirmatory factor analysis (CFA) (Study 2), approximately 200 participants were sought based on a power analysis, suggesting a minimum sample size of 174 participants would provide 80% power for a root mean square error (RMSEA) value of 0.05, and α of .05 (MacCallum, Browne, & Sugawara, 1996) and findings from a simulation study (Flora & Curran, 2004).…”

Child participation in health care (ChiPaC)—Development and psychometric evaluation of a self‐report instrument for children's participation in health care

“…Face validity refers to the clarity and readability of the items and responses and content validity to the adequacy of the instruments' ability to assess what it is presumed to measure (Boateng, Neilands, Frongillo, Melgar‐Quiñonez, & Young, ). In content validity, the items are evaluated for content relevance and representativeness by experts and by the target population (Boateng et al, ; DeVon et al, ).…”

The Need for Inclusion: The relationships between Relational and Collective inclusion needs and psychological well‐and ill‐being

“…Using recommendations in literature for initial scale validation (Boateng, Neilands, Frongillo, Melgar‐Quiñonez, & Young, ; Clark & Watson ; Comrey & Lee, ), we recruited a sample of 200 American adults through Prolific Academic. The duration of the study was 6–7 min and the respondents were paid 0.70 American dollars (approximately 0.64 euros) for their participation.…”