Predicting self-harm at one year in female prisoners: a retrospective cohort study using machine learning

Abstract

Background: Self-harm and suicide are relatively overrepresented in incarcerated populations, especially in female prisons. Identifying those most at risk of significant self-harm could provide opportunities for effective, targeted interventions. Aims: To develop and validate a machine learning-based algorithm capable of achieving a clinically useful level of accuracy when predicting the risk of self-harm in female prisoners. Method: Data were available on 31 variables for 286 female prisoners from a single UK-based prison. This included sociodemographic factors, nature of the index offence, and responses to several psychometric assessment tools used at baseline. At 12-month follow-up any self-harm incidents were reported. A machine learning algorithm (CatBoost) to predict self-harm at one-year was developed and tested. To quantify uncertainty about the accuracy of the algorithm, the model building and evaluation process was repeated 2000 times and the distribution of results summarised. Results: The mean Area Under the Curve (AUC) for the model on unseen (validation) data was 0.92 (SD 0.04). Sensitivity was 0.83 (SD 0.07), specificity 0.94 (SD 0.03), positive predictive value 0.78 (SD 0.08) and the negative predictive value 0.95 (0.02). If the algorithm was used in this population, for every 100 women screened, this would equate to approximately 17 true positives and five false positives. Conclusions: The accuracy of the algorithm was superior to those previously reported for predicting future self-harm in general and prison populations and likely to provide clinically useful levels of prediction. Research is needed to evaluate the feasibility of implementing this approach in a prison setting.