Prediction of eating disorder treatment response trajectories via machine learning does not improve performance versus a simpler regression approach

Abstract

Objective Patterns of response to eating disorder (ED) treatment are heterogeneous. Advance knowledge of a patient's expected course may inform precision medicine for ED treatment. This study explored the feasibility of applying machine learning to generate personalized predictions of symptom trajectories among patients receiving treatment for EDs, and compared model performance to a simpler logistic regression prediction model. Method Participants were adolescent girls and adult women (N = 333) presenting for residential ED treatment. Self‐report progress assessments were completed at admission, discharge, and weekly throughout treatment. Latent growth mixture modeling previously identified three latent treatment response trajectories (Rapid Response, Gradual Response, and Low‐Symptom Static Response) and assigned a trajectory type to each patient. Machine learning models (support vector, k‐nearest neighbors) and logistic regression were applied to these data to predict a patient's response trajectory using data from the first 2 weeks of treatment. Results The best‐performing machine learning model (evaluated via area under the receiver operating characteristics curve [AUC]) was the radial‐kernel support vector machine (AUCRADIAL = 0.94). However, the more computationally‐intensive machine learning models did not improve predictive power beyond that achieved by logistic regression (AUCLOGIT = 0.93). Logistic regression significantly improved upon chance prediction (MAUC[NULL] = 0.50, SD = .01; p <.001). Discussion Prediction of ED treatment response trajectories is feasible and achieves excellent performance, however, machine learning added little benefit. We discuss the need to explore how advance knowledge of expected trajectories may be used to plan treatment and deliver individualized interventions to maximize treatment effects.

A proof-of-concept study applying machine learning methods to putative risk factors for eating disorders: results from the multi-centre European project on healthy eating

“…This finding is consistent with a recent systematic review which found no performance benefit of ML over conventional LR (Christodoulou et al, 2019). Our result is also in line with the ED-specific study by Espel-Huynh et al (2021), which found all SVM models performed similarly well compared to the conventional LR analyses with. The best performing SVM in Espel-Huynh et al's (2021) was the radial-kernel SVM (AUC = 0.94), which was almost identical to the performance of the LR (AUC = 0.93).…”

“…They found that the ML models provided consistently higher prediction accuracy over 1 and 2 years than the conventional LR models (with 19% greater classification accuracy). Another recent study by Espel-Huynh et al (2021) compared ML approaches [support vector machine (SVM) and k-nearest neighbours] to conventional LR models to generate personalised predictions of symptom trajectories among 333 ED patients during the first two weeks of residential treatment. Contrary to Haynos et al's (2020) findings, this study revealed that the ML models did not improve predictive power beyond the one achieved by the LR analyses.…”

Machine learning v. traditional regression models predicting treatment outcomes for binge-eating disorder from a randomized controlled trial

“…Finally, ML may more accurately predict treatment outcomes with time-series predictors v. baseline data alone (e.g. Espel-Huynh et al, 2021; Wang, et al (2021)). Overall, we believe that larger sample sizes, greater numbers of and variability in predictors, and repeated observations are important future directions in predicting eating-disorder treatment outcomes.…”

“…Whereas traditional statistical models predicted self-injurious behaviors barely above chance (Franklin et al, 2017), initial ML studies reported excellent prediction (Fox et al, 2019; Huang et al, 2020; Walsh, Ribeiro, & Franklin, 2017). ML has been applied to eating disorders in several studies (Espel-Huynh et al, 2021; Haynos et al, in press; Sadeh-Sharvit, Fitzsimmons-Craft, Taylor, & Yom-Tov, 2020); ML showed increased predictive accuracy for outcomes relative to traditional models in some (Haynos et al, in press) but not other (Espel-Huynh et al, 2021) studies.…”

Potential benefits and limitations of machine learning in the field of eating disorders: current research and future directions

“…When examining a small number of predictors, current research is mixed on whether ML outperforms traditional analytic approaches in predicting treatment outcomes in clinical settings [ 24 ], including those for eating disorders [ 25 – 27 ]. The main benefit of ML over traditional statistical models is the ability of ML to simultaneously examine large numbers (100s to 1000s) of multimodal predictors and their complex non-linear interactions [ 20 ].…”