Predicting the Risk of Suicide by Analyzing the Text of Clinical Notes

Abstract

We developed linguistics-driven prediction models to estimate the risk of suicide. These models were generated from unstructured clinical notes taken from a national sample of U.S. Veterans Administration (VA) medical records. We created three matched cohorts: veterans who committed suicide, veterans who used mental health services and did not commit suicide, and veterans who did not use mental health services and did not commit suicide during the observation period (n = 70 in each group). From the clinical notes, we generated datasets of single keywords and multi-word phrases, and constructed prediction models using a machine-learning algorithm based on a genetic programming framework. The resulting inference accuracy was consistently 65% or more. Our data therefore suggests that computerized text analytics can be applied to unstructured medical records to estimate the risk of suicide. The resulting system could allow clinicians to potentially screen seemingly healthy patients at the primary care level, and to continuously evaluate the suicide risk among psychiatric patients.

Into the Bowels of Depression: Unravelling Medical Symptoms Associated with Depression by Applying Machine-Learning Techniques to a Community Based Population Sample

“…ML techniques have been used in the area of big data informatics in mental health. For example, text analysis [45] and regression models [46] have been used to predict the risk of suicide from clinical notes, but these techniques have not previously been used to investigate the relationship between depression and medical symptoms using epidemiological community based population data. The visual simplification of complex medical symptom data into clusters, using SOM, allows the researcher to easily identify the strength of the similarities across the map.…”

Social media, big data, and mental health: current advances and ethical implications

“…As is to be expected, the field has moved on significantly since the development of these early chatbots. Recent work uses sophisticated NLP and ML methods to, for instance, assess suicide risk in pediatric populations based on writing samples [29], predict depression severity and optimal treatment based on narrative text derived from Electronic Health Records [30], identify linguistic features characteristic of early stage dementia [31], and predict the suicide risk of active duty military personnel based on Electronic Health Record data [32]. In parallel with these advances in NLP, there is a rich tradition in the psychology domain (exemplified by Pennebaker [33]) of using carefully developed and validated lexicons organized into various categories (e.g.…”

The use of electronic health records for psychiatric phenotyping and genomics

“…A recent comprehensive meta-analysis found that the predictive ability of known risk factors is weak and has not improved over the past 50 years. [Franklin and others 2016] A number of investigators have used EHR data to identify individuals at risk for suicidal behavior, but studies have been limited by relatively small samples [Baca-Garcia and others 2006; Poulin and others 2014], short follow-up times [Tran and others 2014], specialized populations [Kessler and others 2015], or have not reported performance metrics. [Ilgen and others 2009] We recently developed risk prediction models for suicidal behavior by leveraging the high-dimensional clinical data available in the EHR and using time-varying data across an extended period of follow-up.…”