Development and Validation of a Machine Learning Prediction Model of Posttraumatic Stress Disorder After Military Deployment

Abstract

ImportanceMilitary deployment involves significant risk for life-threatening experiences that can lead to posttraumatic stress disorder (PTSD). Accurate predeployment prediction of PTSD risk may facilitate the development of targeted intervention strategies to enhance resilience.ObjectiveTo develop and validate a machine learning (ML) model to predict postdeployment PTSD.Design, Setting, and ParticipantsThis diagnostic/prognostic study included 4771 soldiers from 3 US Army brigade combat teams who completed assessments between January 9, 2012, and May 1, 2014. Predeployment assessments occurred 1 to 2 months before deployment to Afghanistan, and follow-up assessments occurred approximately 3 and 9 months post deployment. Machine learning models to predict postdeployment PTSD were developed in the first 2 recruited cohorts using as many as 801 predeployment predictors from comprehensive self-report assessments. In the development phase, cross-validated performance metrics and predictor parsimony were considered to select an optimal model. Next, the selected model’s performance was evaluated with area under the receiver operating characteristics curve and expected calibration error in a temporally and geographically distinct cohort. Data analyses were performed from August 1 to November 30, 2022.Main Outcomes and MeasuresPosttraumatic stress disorder diagnosis was assessed by clinically calibrated self-report measures. Participants were weighted in all analyses to address potential biases related to cohort selection and follow-up nonresponse.ResultsThis study included 4771 participants (mean [SD] age, 26.9 [6.2] years), 4440 (94.7%) of whom were men. In terms of race and ethnicity, 144 participants (2.8%) identified as American Indian or Alaska Native, 242 (4.8%) as Asian, 556 (13.3%) as Black or African American, 885 (18.3%) as Hispanic, 106 (2.1%) as Native Hawaiian or other Pacific Islander, 3474 (72.2%) as White, and 430 (8.9%) as other or unknown race or ethnicity; participants could identify as of more than 1 race or ethnicity. A total of 746 participants (15.4%) met PTSD criteria post deployment. In the development phase, models had comparable performance (log loss range, 0.372-0.375; area under the curve range, 0.75-0.76). A gradient-boosting machine with 58 core predictors was selected over an elastic net with 196 predictors and a stacked ensemble of ML models with 801 predictors. In the independent test cohort, the gradient-boosting machine had an area under the curve of 0.74 (95% CI, 0.71-0.77) and low expected calibration error of 0.032 (95% CI, 0.020-0.046). Approximately one-third of participants with the highest risk accounted for 62.4% (95% CI, 56.5%-67.9%) of the PTSD cases. Core predictors cut across 17 distinct domains: stressful experiences, social network, substance use, childhood or adolescence, unit experiences, health, injuries, irritability or anger, personality, emotional problems, resilience, treatment, anxiety, attention or concentration, family history, mood, and religion.Conclusions and RelevanceIn this diagnostic/prognostic study of US Army soldiers, an ML model was developed to predict postdeployment PTSD risk with self-reported information collected before deployment. The optimal model showed good performance in a temporally and geographically distinct validation sample. These results indicate that predeployment stratification of PTSD risk is feasible and may facilitate the development of targeted prevention and early intervention strategies.

Parvalbumin interneuron activity in autism underlies susceptibility to PTSD-like memory formation

“…Our results suggest that everyday life situations could be experienced as traumatic in ASD patients, as previously suggested in clinical studies. 8 , 48 As new methods of detection emerge, 49 our study calls for better use of predictive tools that enable efficient risk assessment and early interventions of PTSD among those likely to experience trauma, such as people with autism. Timely detection appears essential since PTSD worsens the core ASD traits and is strongly associated with various psychiatric comorbidities and suicide.…”

Parvalbumin Interneuron Activity Underlies Vulnerability to Post-Traumatic Stress Disorder in Autism

“…Our results indeed support that everyday life situations could be experienced as traumatic in ASD patients, as previously suggested from clinical studies 8,46 . As new methods of detection emerge 47 , our study calls for better use of predictive tools that enable efficient risk assessment and early interventions of PTSD among those likely to experience trauma, such as patients with autism. Timely detection appears essential since PTSD worsens the core symptoms of ASD and is strongly associated with various psychiatric comorbidities and suicide 48 .…”

Development and validation of a machine learning model using electronic health records to predict trauma- and stressor-related psychiatric disorders after hospitalization with sepsis

“…This is problematic because complete case analyses are only valid when data are missing completely at random (MCAR) [ 25 ], which is an unrealistically strong assumption that ignores the potential for attrition bias and can result in inaccurate performance estimates. Consistent with more recent prediction models [ 11 , 26 ], we accounted for these encounters by using inverse probability of censoring weights, a robust approach for including cases when outcomes are missing at random (MAR; a less stringent assumption than MCAR); this approach leads to less biased performance estimates compared to complete-case approaches [ 27 ]. A stacked ensemble of ML algorithms (i.e., the approach used for the TSRD outcome model described in Statistical Analyses) was used to estimate the probability of having an observed outcome using all the pre-discharge predictors, and the inverse of these probabilities were used as weights in the TSRD prediction models.…”

“…To select a final model, we considered log-loss (lower is better) and area under the receiver operating characteristics (ROC) curve (AUC; higher is better, and an AUC of 0.50 indicates random or chance-level performance); these performance metrics were estimated in the development sample using 10-fold cross-validation (8 folds for the core-predictor GBM since two of the folds were used to identify core predictors). Consistent with prior prediction models, patients who had a pre-existing trauma- and stressor-related psychiatric diagnoses were not excluded from the sample [ 11 , 12 , 24 , 26 ]. Instead, a univariate generalized linear model that used pre-existing TSRD diagnosis as the sole predictor of TSRD in the follow-up phase served as an informative benchmark: Y i = β 0 + β×(pre-existing PTSD) i + ε i The purpose of testing whether models outperformed this benchmark is two-fold.…”

“…Second, this benchmark model captures how much of the predictive accuracy is driven by pre-existing diagnosis. Similar benchmarks have been used in other predictive models of psychiatric outcomes after trauma exposure [ 11 , 12 , 24 , 26 ]. Given that simpler models may be easier to implement, we also took predictor parsimony into account such that a model with fewer predictors and comparable performance would be favored.…”