Applying generalizability theory methods to assess continuity and change on the Adolescent Quality of Life-Mental Health Scale (AQOL-MHS)

Abstract

Purpose The Adolescent Quality of Life Mental Health Scale (AQOL-MHS) is designed to measure quality of life in clinical samples of Latino adolescents aged 12–18 years. Initial findings support its reliability, validity and conceptual model for use of its three domains (Emotional Regulation, Self-Concept, and Social Context). Our current study tests the usefulness of the AQOL-MHS for tracking changes in HRQOL during the course of service use. Methods Three waves of data were collected from 59 participants who were recruited from mental health clinics, where waves were spaced four months apart. Participants were receiving services at baseline assessment and were tracked for follow-up appointments regardless of treatment status. We analyze conventional reliability statistics for individual differences (e.g. Cronbach's alpha and test-retest correlations) and to estimate the reliability of change, we carried out a variance decomposition analysis. Results The psychometric analyses from Chavez et al. [1] were replicated with comparable results. A Generalizability Theory (GT) analysis revealed that the AQOL-MHS domains can measure change reliably. The reliability estimates varied from .65 to .78. Although there was reliable change at the individual level, on the average the AQOL-MHS means improved only slightly over time. Conclusions The reliability of change for all three scales in the AQOL-MHS was acceptable, but consistently lower than the Cronbach’s alpha values for each wave. Future work will use this approach to adapt our current measure to provide a better reliability of within person change and thus broaden its applicability for prospective use.

Distinguishing transient versus stable aspects of depression in New Zealand Pacific Island children using Generalizability Theory

“…Lakes and Hoyt (2009) have argued that G Theory is a particularly useful conceptual framework for measuring the stable and transient components of a latent construct among adolescents where development is paramount. A recent paper by Chavez et al, (2016) demonstrated this approach with the Adolescent Quality of Life-Mental Health Scale (AQOL-MHS) in adolescents attending mental health clinics assessed on three occasions over eight months. The focus of that study was on the reliability of change scores and the authors used a method developed by Cranford et al (2006) based on G Theory to compare conventional estimates of reliability (Cronbach's alpha, test-retest) with GT estimates.…”

Psychometric properties of instruments assessing exercise in patients with eating disorders: a systematic review

“…While all measures identified in this systematic review were constructed using classical test theory, there are other measurement models that might be more suitable to measuring compulsive exercise such as item response theory and generalizability theory, which would allow for separation of systematic from unsystematic errors, and for both norm-referenced and criterion-referenced outcomes [ 61 , 62 ]. Future instruments may benefit from a different theoretical underpinning to explore alternative measurement models.…”

Validation of the Simplified Chinese Version of FACT-Hep for Patients with Hepatocellular Carcinoma Based on Combinations of Classical Test Theory and Generalizability Theory

“…Besides, we also applied Generalizability Theory (G theory) to investigate the score dependability of the FACT-Hep, which addresses the dependability of measurements and the simultaneous estimation of multiple sources of variance including interactions [28][29][30]. We employed a random effects design for both the G-study and D-study in measurement mode to estimate the variance components and dependability coe cients using a one-facet crossed design: persons (p) by items (i), represented as p x i, where the patients as the object of measurement and not a source of error and not considered a facet, but the items as one facet of measurement error.…”