Machine Learning–Based Suicide Risk Prediction Model for Suicidal Trajectory on Social Media Following Suicidal Mentions: Independent Algorithm Validation

Abstract

Background Previous efforts to apply machine learning–based natural language processing to longitudinally collected social media data have shown promise in predicting suicide risk. Objective Our primary objective was to externally validate our previous machine learning algorithm, the Suicide Artificial Intelligence Prediction Heuristic (SAIPH), against external survey data in 2 independent cohorts. A second objective was to evaluate the efficacy of SAIPH as an indicator of changing suicidal ideation (SI) over time. The tertiary objective was to use SAIPH to evaluate factors important for improving or worsening suicidal trajectory on social media following suicidal mention. Methods Twitter (subsequently rebranded as X) timeline data from a student survey cohort and COVID-19 survey cohort were scored using SAIPH and compared to SI questions on the Beck Depression Inventory and the Self-Report version of the Quick Inventory of Depressive Symptomatology in 159 and 307 individuals, respectively. SAIPH was used to evaluate changing SI trajectory following suicidal mentions in 2 cohorts collected using the Twitter application programming interface. Results An interaction of the mean SAIPH score derived from 12 days of Twitter data before survey completion and the average number of posts per day was associated with quantitative SI metrics in each cohort (student survey cohort interaction β=.038, SD 0.014; F4,94=3.3, P=.01; and COVID-19 survey cohort interaction β=.0035, SD 0.0016; F4,493=2.9, P=.03). The slope of average daily SAIPH scores was associated with the change in SI scores within longitudinally followed individuals when evaluating periods of 2 weeks or less (ρ=0.27, P=.04). Using SAIPH as an indicator of changing SI, we evaluated SI trajectory in 2 cohorts with suicidal mentions, which identified that those with responses within 72 hours exhibit a significant negative association of the SAIPH score with time in the 3 weeks following suicidal mention (ρ=–0.52, P=.02). Conclusions Taken together, our results not only validate the association of SAIPH with perceived stress, SI, and changing SI over time but also generate novel methods to evaluate the effects of social media interactions on changing suicidal trajectory.