The Loneliness Questionnaire–Short Version: An Evaluation of Reverse-Worded and Non-Reverse-Worded Items Via Item Response Theory

Abstract

Although reverse-worded items have often been incorporated in scale construction to minimize the effects of acquiescent reporting biases, some researchers have more recently begun questioning this approach and wondering whether the advantages associated with incorporating reverse-worded items is worth the complexities that they bring to measures (e.g., Brown, 2003 ; Marsh, 1996 ). In this study, we used item response theory (IRT) to determine whether there is statistical justification to eliminate the reverse-worded items (e.g., "I have lots of friends") from the Loneliness Questionnaire (LQ; Asher, Hymel, & Renshaw, 1984) and retain only the non-reverse-worded items (e.g., "I'm lonely") to inform the provision of a shortened LQ version. Using a large sample of children (Grades 2-7; n = 6,784) and adolescents (Grades 8-12; n = 4,941), we examined the psychometric properties of the 24-item LQ and found support for retaining the 9 non-reverse-worded LQ items to make up a shortened measure of loneliness in youth. We found that the non-reverse-worded items were associated with superior psychometric properties relative to the reverse-worded items with respect to reliability and IRT parameters (e.g., discrimination and item information). A 3-point Likert-type scale was also found to be more suitable for measuring loneliness across both children and adolescents compared to the original 5-point scale. The relative contributions of reverse-worded and non-reverse-worded items in scale development for youth instruments are also discussed.

Validation of the Anxiety Scale for Pregnancy in a Sample of Iranian Women

“…The lower value of alpha for baby factor may be due to the fact that it consists of both negatively and positively worded items. This result is consistent with recent evidences (37,38) against the previously recommended method of using the mixed item format (i.e. including both positively and negatively worded items) in scales (39).…”

A comparison of self-report measures of psychopathy among nonforensic samples using item response theory analyses.

“…Our finding that these reverse-scored items may have limited ability in distinguishing among respondents with varying levels of the latent trait adds to the growing body of literature reporting problems with negatively worded items in personality assessment (e.g., Crego & Widiger, 2014; Ebesutani et al, 2012; Lindwall et al, 2012; Ray, Frick, Thornton, Steinberg, & Cauffman, 2016; Rodebaugh, Woods, & Heimberg, 2007; Stansbury, Ried, & Velozo, 2006). These results raise two practical questions.…”

Increasing the Sensitivity of Measures to Change

“…Attempting to minimize biases arising from perseveration, acquiescence, or naive hypotheses, developers of measures frequently include reverse-worded items or scale anchors (Cox, 1980; Ebesutani, et al, 2012; Schwarz, 1999). Schwarz emphasized the importance of the cognitive processes involved in survey methodology.…”

“…The central concern of prevention researchers working with small samples is poor statistical power for detecting a true preventive effect (Fok, Henry, & Allen, this issue), as power is a function of the effect size and the sample size. Detection of an effect that involves change in an outcome due to intervention or developmental effects is mediated by the ability of measures to accurately assess change, a characteristic that is variously termed “responsiveness” or “sensitivity to change” (Ebesutani, et al, 2012). However, little attention is paid in prevention research to responsiveness or sensitivity to change.…”

“…Measures that distinguish among different individuals at a single point in time with high reliability may have limited utility when asked to assess change within individuals over time (Hays & Hadorn, 1992; West, Celio, Henry, & Pavuluri, 2010). Anchors or rating scales chosen for their ability to rank individuals may not be equally suited to detecting change (Ebesutani et al, 2012; Guyatt, Deyo, Charlson, Levine, & Mitchell, 1989). If respondents have difficulty differentiating among anchors close in meaning, such as “occasionally” and “sometimes,” noise will be added to the measurement, possibly resulting poorer ability to detect change.…”