Measuring Symptoms of Psychopathology in Zambian Orphans and Vulnerable Children: Scale Validation and Psychometric Evaluation

Abstract

There is a paucity of validated mental health measures for assessing psychological well-being among HIV-affected youth. We sought to explore the psychometric properties and validity of the Achenbach Youth Self-Report and Child Posttraumatic Stress Disorder Symptom Scale among orphans and vulnerable children (OVC) living in Lusaka, Zambia. These scales were administered to 210 OVC aged 13 to 17 years via audio computer-assisted self-interview. Confirmatory factor analysis was used to assess scale structure, Cronbach's alpha for internal consistency, and correlations between scales related to mental or psychosocial health for construct validity. A known-groups validation was conducted using local identifications of youth with and without significant psychosocial problems, and test-retest reliability was assessed. Scales exhibited good internal reliability (α > .80), adequate criterion validity (area under the curve > .70), and moderate test-retest reliability (.62-.68). Findings support the utility of these symptom scales for identifying OVC experiencing significant psychosocial problems in Zambia.

When less is more: reducing redundancy in mental health and psychosocial instruments using Item Response Theory

“…We based item selection on the same considerations as above for Thailand. Using the other half of the instrument validation study data (Murray et al ., 2018 b ), we compared the performance of a shortened CPSS to the original version across a number of psychometrics: (1) Cronbach's α ; (2) score distributions (i.e. means and standard deviations); (3) criterion validity comparing average scores on the original CPSS and short CPSS for PTSD cases and non-cases (Kane et al ., 2018; Murray et al ., 2018 b ); and (4) clinical utility comparing Area Under the Curves (AUCs) using our criterion.…”

“…We used IRT to generate shortened scales and then tested how well the shortened scales measured change over time in the enrolled sample ( N = 347). The second source of data is from HIV-affected adolescents (ages 13–17) in Zambia who completed assessments either as part of a cross-sectional instrument development study ( N = 210) (Kane et al ., 2018; Murray et al ., 2018 b ) or during screening for an RCT of trauma-focused cognitive behavioral therapy ( N = 610; NCT02054780) that used the measures tested in the instrument development study. These samples were selected due to their purposive sampling approach, which was intended to include people with and without mental health problems representing a range of underlying disease severity.…”

“…Scores were calculated by taking the mean of all responses for each participant. Results from a previously conducted instrument development and validation study with the CPSS in Zambia found that the measure had good internal reliability (Cronbach's α = 0.93), adequate test–retest reliability (0.68), and strong criterion validity (the measure significantly discriminated between PTSD cases and non-cases at p < 0.05) (Murray et al ., 2018 b ).…”

“…For Zambia, participants in each study were distinct but drawn from the same source population of adolescents who exhibited HIV risk behaviors and met the WHO criteria for orphan or vulnerable child (Kane et al ., 2018; Murray et al ., 2018 b ). Item selection for the CPSS based on GRMs was conducted with a random sample of 50% of the data from the instrument validation study.…”

“…Using the other half of the instrument validation study data (Murray et al ., 2018 b ), we compared the performance of a shortened CPSS to the original version across a number of psychometrics: (1) Cronbach's α ; (2) score distributions (i.e. means and standard deviations); (3) criterion validity comparing average scores on the original CPSS and short CPSS for PTSD cases and non-cases (Kane et al ., 2018; Murray et al ., 2018 b ); and (4) clinical utility comparing Area Under the Curves (AUCs) using our criterion. We tested the shortened CPSS performance in the baseline RCT data by comparing the relative strength of correlations between the shortened CPSS version and the full version with three external measures: a locally-developed scale of functional impairment (Murray et al ., 2015), and the Youth Self Report (YSR), which includes sub-scales of both internalizing and externalizing symptoms (Achenbach, 1991).…”