Prediction models for child and adolescent mental health: A systematic review of methodology and reporting in recent research

Abstract

Background There has been a rapid growth in the publication of new prediction models relevant to child and adolescent mental health. However, before their implementation into clinical services, it is necessary to appraise the quality of their methods and reporting. We conducted a systematic review of new prediction models in child and adolescent mental health, and examined their development and validation. Method We searched five databases for studies developing or validating multivariable prediction models for individuals aged 18 years old or younger from 1 January 2018 to 18 February 2021. Quality of reporting was assessed using the Transparent Reporting of a multivariable prediction models for Individual Prognosis Or Diagnosis checklist, and quality of methodology using items based on expert guidance and the PROBAST tool. Results We identified 100 eligible studies: 41 developing a new prediction model, 48 validating an existing model and 11 that included both development and validation. Most publications (k = 75) reported a model discrimination measure, while 26 investigations reported calibration. Of 52 new prediction models, six (12%) were for suicidal outcomes, 18 (35%) for future diagnosis, five (10%) for child maltreatment. Other outcomes included violence, crime, and functional outcomes. Eleven new models (21%) were developed for use in high‐risk populations. Of development studies, around a third were sufficiently statistically powered (k = 16%, 31%), while this was lower for validation investigations (k = 12, 25%). In terms of performance, the discrimination (as measured by the C‐statistic) for new models ranged from 0.57 for a tool predicting ADHD diagnosis in an external validation sample to 0.99 for a machine learning model predicting foster care permanency. Conclusions Although some tools have recently been developed for child and adolescent mental health for prognosis and child maltreatment, none can be currently recommended for clinical practice due to a combination of methodological limitations and poor model performance. New work needs to use ensure sufficient sample sizes, representative samples, and testing of model calibration.

Preschool development, temperament and genetic liability as early markers of childhood ADHD: A cohort study

“…Future research may also look at the use of the identified early markers and precursors to develop a prediction model for later ADHD. However, the use of prediction models within child and adolescent psychiatry has been shown to have limited utility, in part due to methodological issues such as sample size and replicability, which would have been seen in this study had we developed such a model (Larsson, 2021;Senior et al, 2021).…”

Challenges and opportunities for the early identification of ADHD across multiple levels of analysis – A reflection on Tobarra‐Sanchez et al. (2022)

“…Although beyond the scope of the present paper, the work by Tobarra-Sanchez et al also (2022) emphasizes the need for the field to go beyond individual predictors and begin to develop and test prognostic diagnostic models (Larsson, 2021;Senior et al, 2021) which have direct translational implications. For example, prognostic…”

Progress and barriers to the implementation of prediction modelling in child and adolescent mental health—A commentary on Senior et al. (2021)

“…One of the most striking findings reported by Senior et al (2021) is that only 2 of the 100 eligible studies demonstrated a low 'risk of bias', driven both by inappropriate analytic strategies and insufficient reporting of key study characteristics. Risk of bias was evaluated across four domains (participants, predictors, outcome and analysis) using the Prediction model Risk Of Bias ASsessment Tool (PROBAST; Wolff et al, 2019).…”