Prediction Tool for Individual Outcome Trajectories Across the Next Year in First-Episode Psychosis in Coordinated Specialty Care

Abstract

ImportanceIn coordinated specialty care (CSC) settings for people with a first episode of psychosis, the development of reliable, validated individual-level prediction tools for key outcomes may be informative for shared clinician and client decision-making.ObjectiveTo develop an individual-level prediction tool using machine-learning methods that predicts a trajectory of education/work status or psychiatric hospitalization outcomes over a client’s next year of quarterly follow-up assessments. Additionally, to visualize these predictions in a way that is informative to clinicians and clients.Design, Setting, and ParticipantsIndividual-level data were collected for all patients enrolled in the OnTrackNY program at enrollment and at quarterly follow-ups using standardized forms. The OnTrackNY program, a network of CSC sites in New York State, provides person-centered, recovery-oriented, and evidence-based psychosocial and pharmaceutical interventions to individuals aged 16 to 30 years with recent-onset (<2 years) nonaffective psychosis. Although data collection is ongoing, data for this study were collected from October 2013 to December 2018, and the time frame for analysis was July 2020 to May 2021. Data were separated into a training/cross-validation set to perform internally validated model development and a separate holdout test set (~20% of the sample) for external validation. Random probability forest models were developed to predict individual-level trajectories of outcomes.ExposuresForty-three individual-level demographic and clinical features collected at enrollment in OnTrackNY, 25 of which were time-varying and updated at quarterly follow-up assessments, and 13 site-level demographic and economic census variables.Main Outcomes and MeasuresIndividual-level education and/or employment status and psychiatric hospitalization trajectories at quarterly follow-up periods across the first 2 years of CSC.ResultsThe total study sample consists of 1298 individuals aged 16 to 30 years and included 341 women (26.3%), 949 men (73.1%), and 8 (<1%) with another gender. Prediction models performed well for 1-year trajectories of education/work across all validation sets, with areas under the receiver operating characteristic curve (AUCs) ranging from 0.68 (95% CI, 0.63-0.74) to 0.88 (95% CI, 0.81-0.96). Predictive accuracy for psychiatric hospitalization 3 months ahead reached AUC above 0.70; moreover, predictions of future psychiatric hospitalizations at 6 months and beyond were consistently poor, with AUCs below 0.60. Given the good externally validated performance for predicting education/work, a prototype interactive visualization tool displaying individual-level education/work trajectories and related features was developed.Conclusions and RelevanceThis study suggests that accurate prediction tools can be developed for outcomes in people with first-episode psychosis, which may help inform shared clinician/client decision-making. Future work should study the effectiveness of its deployment, including proper communication to inform shared clinician/client decision-making in the context of a learning health care system. At present, more work is needed to develop better performing prediction models for future psychiatric hospitalizations before any tool is recommended for this outcome.

Deciphering the Mosaic of Therapeutic Potential: A Scoping Review of Neural Network Applications in Psychotherapy Enhancements

“…Prediction of patients' clinical trajectories, especially in the field of psychosis, is an eminent avenue of research in psychiatry. Such approaches could lead to more personalized treatments [36]. As identified in this scoping review, the use of NNs for predicting outcomes was brought up in Gori's study of 2010, which stated that such use of NNs could help the therapist with their decision making, and help in selecting the appropriate treatment for the patients [27].…”

Psychosis Prognosis Predictor: A Continuous and Uncertainty-Aware Prediction of Treatment Outcome in First-Episode Psychosis

“…In psychiatry research, machine learning techniques are increasingly being used, particularly in psychotic disorders 3 . To the best of our knowledge, ten studies [4][5][6][7][8][9][10][11][12][13] examined illness progress in existing psychotic disorders, each predicting different outcomes. Of these, only one study 4 aimed to predict treatment response.…”