Multiple informant average integration of ADHD symptom ratings predictive of concurrent and longitudinal impairment.

Abstract

To date, there remains no consensus about the best evidence-based method for integrating multiple informant data in the diagnosis of Attention-Deficit/Hyperactivity Disorder (ADHD). Several approaches exist, including the psychometrically sound approach of averaging scores, as well as the use of "OR" and "AND" algorithms, which are still commonly used in research. The current study tested these major integration methods in their concurrent and longitudinal prediction of clinician-rated impairment, teacherrated academic, and parent-and self-rated social skill ratings in children overrecruited for ADHD across a 6-year span from childhood to adolescence. The sample included a total of 800 children, 480 with ADHD, ages 6 to 13, who completed a "gold standard" assessment of ADHD and associated impairment. Overall, the "OR," "AND," and average integration approaches showed significantly high interrelations with one another (r range from .78 to .96) and were all significantly and strongly related to impairment measures concurrently and longitudinally. Multivariate regressions demonstrated that the average integration approach concurrently and longitudinally out predicted the other two approaches. Results demonstrated that the average approach slightly outperformed the other two in its prediction of concurrent and longitudinal clinician-rated impairment, teacher-rated academic skills, and parent-and self-rated child social skills across childhood and adolescence. Evidence-based assessment integration of parent and teacher ratings of ADHD in childhood might best utilize an averaging approach, as it is most related to later impairment ratings, particularly if such findings are replicated by other groups. Public Significance StatementAn average approach to integrating parent and teacher ratings of ADHD slightly outperforms other, more complicated integration approaches in prediction of later clinician-rated impairment, teacher-rated academic skills, and parent-and self-rated social skills. Therefore, average integration of ADHD symptom ratings may be the best and easiest integration approach for use in clinical practice.

Conceptual, methodological, and measurement factors that disqualify use of measurement invariance techniques to detect informant discrepancies in youth mental health assessments

“…Measurement invariance techniques are not alone in emphasizing common variance. The majority of analytic techniques currently used by researchers seeking to integrate or model multi-informant data in youth mental health emphasize common variance and treat unique variance as error, including composite scores (e.g., Martel et al, 2021 ), many applications of structural equations modeling (e.g., Eid et al, 2008 ; Bauer et al, 2013 ; Howe et al, 2019 ; Watts et al, 2022 ), and combinational algorithms (e.g., AND rule; Baumgartner et al, 2021 ).…”

Generalizable prediction of childhood ADHD symptoms from neurocognitive testing and youth characteristics

“…Given the importance of considering multiple raters when assessing ADHD symptoms [35][36][37] , we included participants who had complete parent and teacher ratings on key ADHD-relevant scales (described below) at the same timepoint. At baseline, 3,960 children met this criterion while 5,915 children met this criterion for the 1-year follow-up.…”

“…In the current study, we leverage ABCD to develop and test generalizable machine learning models for predicting ADHD symptoms in independent data using neurocognitive testing and features from other relevant measurement domains (demographics, geocoding, and child reports of personality, family structure, and social context). We index ADHD symptoms using a multi-rater approach, considered the gold standard for minimizing rater biases and measuring impairments across contexts [35][36][37] , and predict this symptom index using both comprehensive predictive modeling methods, which account for the latent structure of covariation among predictors, and sparse modeling methods that only include features with unique predictive information. We use leave-one-site-out cross-validation to assess model accuracy in unseen data and quantify the predictive contributions of neurocognitive testing versus other measurement domains.…”

Integrating multi-informant reports of youth mental health: A construct validation test of Kraemer and colleagues’ (2003) Satellite Model

“…Consider examples of the most commonly implemented integrative approaches. For instance, the composite score approach involves computing the sum or average of the informants’ scores (see Martel et al, 2021 ). Its key underlying assumption is that greater agreement among informants’ scores signals greater severity of the mental health concerns about which informants provide reports (e.g., De Los Reyes et al, 2016 ).…”