Classifying Characteristics of Opioid Use Disorder From Hospital Discharge Summaries Using Natural Language Processing

Abstract

BackgroundOpioid use disorder (OUD) is underdiagnosed in health system settings, limiting research on OUD using electronic health records (EHRs). Medical encounter notes can enrich structured EHR data with documented signs and symptoms of OUD and social risks and behaviors. To capture this information at scale, natural language processing (NLP) tools must be developed and evaluated. We developed and applied an annotation schema to deeply characterize OUD and related clinical, behavioral, and environmental factors, and automated the annotation schema using machine learning and deep learning-based approaches.MethodsUsing the MIMIC-III Critical Care Database, we queried hospital discharge summaries of patients with International Classification of Diseases (ICD-9) OUD diagnostic codes. We developed an annotation schema to characterize problematic opioid use, identify individuals with potential OUD, and provide psychosocial context. Two annotators reviewed discharge summaries from 100 patients. We randomly sampled patients with their associated annotated sentences and divided them into training (66 patients; 2,127 annotated sentences) and testing (29 patients; 1,149 annotated sentences) sets. We used the training set to generate features, employing three NLP algorithms/knowledge sources. We trained and tested prediction models for classification with a traditional machine learner (logistic regression) and deep learning approach (Autogluon based on ELECTRA's replaced token detection model). We applied a five-fold cross-validation approach to reduce bias in performance estimates.ResultsThe resulting annotation schema contained 32 classes. We achieved moderate inter-annotator agreement, with F1-scores across all classes increasing from 48 to 66%. Five classes had a sufficient number of annotations for automation; of these, we observed consistently high performance (F1-scores) across training and testing sets for drug screening (training: 91–96; testing: 91–94) and opioid type (training: 86–96; testing: 86–99). Performance dropped from training and to testing sets for other drug use (training: 52–65; testing: 40–48), pain management (training: 72–78; testing: 61–78) and psychiatric (training: 73–80; testing: 72). Autogluon achieved the highest performance.ConclusionThis pilot study demonstrated that rich information regarding problematic opioid use can be manually identified by annotators. However, more training samples and features would improve our ability to reliably identify less common classes from clinical text, including text from outpatient settings.

Predicting Blood Pressure After Nitroglycerin Infusion Dose Titration in Critical Care Units

“…27 It has also been used to predict the course of treatment for various conditions, such as heart failure and diabetes. Additionally, it has been used to classify characteristics of opioid use disorder from hospital discharge summaries 28 and forecasting walking assistance rehabilitation to guide clinical care pathways after stroke. 29 One of the primary advantages of AutoML is its speed.…”

Novel digital approaches to the assessment of problematic opioid use

“…However, risk factors can stem from five basic sources (Fig. 1): Lifetime and current psychiatric disease comorbidities that include both mental disorders and substance use disorders (including previous opioid use) [32][33][34], medical histories from EHR data and clinical notes [35,36], environmental and societal factors that include demographics and personal histories [28], digital footprints including social media and biometrics [37][38][39], and omics data which comprise genetic, epigenetic, transcriptomic, and other large-scale biological data [19][20][21][22][23][24][25][26][27]. In Fig.…”

“…Clinical notes can also be a robust source of risk indicators for POU as they can capture critical pieces of information not available in structured EHR data. During a clinical encounter, a patient may discuss topics with a healthcare professional that may be associated with risk of POU development but not be captured in specific EHR data entry fields [36]. If this information is recorded electronically, natural language processing (NLP) approaches can identify features from typed language that can assist in the development of risk assessment protocols [64,90].…”

“…A recent study highlights one such improvement. The researchers performed manual reviews of hospital discharge summaries and identified several text classes describing potential POU [36]. Annotated sentences were used to generate features using the open-source knowledge bases Empath [126], the Unified Medical Language System [127], and PyConText [128].…”

Developing a Framework to Infer Opioid Use Disorder Severity From Clinical Notes to Inform Natural Language Processing Methods: Characterization Study

“…Patients with chronic pain with opioid prescriptions are at increased risk for opioid misuse [ 16 , 17 ], but only focusing on such populations may exclude patient populations with illicit opioid use or otherwise outside of chronic pain treatment. Previous research indicates clinical notes provide a source of rich information that could improve efforts to identify and characterize OUD [ 12 , 18 , 19 ].…”