MADEx: A System for Detecting Medications, Adverse Drug Events, and Their Relations from Clinical Notes

Abstract

Introduction Early detection of Adverse Drug Events (ADEs) from Electronic Health Records (EHRs) is an important, challenging task to support pharmacovigilance and drug safety surveillance. A well-known challenge to use clinical text for detection of ADEs is that much of the detailed information is documented in a narrative manner. Clinical Natural Language Processing (NLP) is the key technology to extract information from unstructured clinical text. Objective We present a machine learning-based clinical NLP system - MADEx for detecting medications, ADEs and their relations from clinical notes. Methods We developed a Recurrent Neural Network (RNN) model using Long Short-Term Memory (LSTM) strategy for clinical Name Entity Recognition (NER) and compared it with a baseline Conditional Random Fields (CRFs). We developed a modified training strategy for RNN, which outperformed the widely used early stop strategy. For relation extraction, we compared Support Vector Machines (SVMs) and Random Forests on single-sentence relations and cross-sentence relations. We also developed an integrated pipeline to extract entities and relations together by combining RNN and SVMs. Results MADEx achieved top three best performance (F1-score of 0.8233) for clinical NER in the 2018 Medication and Adverse Drug Events (MADE1.0) challenge. The post-challenge evaluation showed that the relation extraction module and integrated pipeline (identify entity and relation together) of MADEx are comparable to the best systems developed in this challenge. Conclusion This study demonstrated the efficiency of deep learning methods for automatic extraction of medications, ADEs, and their relations from clinical text to support pharmacovigilance and drug safety surveillance.

Overview of the First Natural Language Processing Challenge for Extracting Medication, Indication, and Adverse Drug Events from Electronic Health Record Notes (MADE 1.0)

“…For the first task, we found no statistically significant difference between the first [58] and second [59] system. However, the third system [60] was statistically significantly different from both Wunnava et al [58] and Dandala et al [59]. For task2, all differences between the top three teams were statistically significant.…”

Role of Artificial Intelligence in Patient Safety Outcomes: Systematic Literature Review

“…Studies in this category retrieved data from different repositories such as DrugBank, Side Effect Resource, the Food and Drug Administration (FDA)’s adverse event reporting system, University of Massachusetts Medical School, Observational Medical Outcomes Partnership database, and Human Protein-Protein Interaction database to identify adverse drug interactions and reactions that can potentially negatively influence patient health [ 86 - 88 , 101 , 102 , 105 - 107 , 110 ]. Some studies also used AI to predict drug interactions by analyzing EHR data [ 88 ], unstructured discharge notes [ 90 ], and clinical charts [ 99 , 104 ]. One study also used AI to identify drugs that were withdrawn from the commercial markets by the FDA [ 100 ].…”

Medical Information Extraction in the Age of Deep Learning

“…Wei et al [62] Tao et al [71] Gligic et al [63] Li et al [66] Dandala et al [70] Chapman et al [72] Wunnava et al [67] Gligic et al [63] Wei et al [62] Tao et al [71] Wunnava et al [67] Chapman et al [72] Li et al [66] Dandala et al [70] Wei et al [62] Gligic et al [63] Tao et al [71] Wunnava et al [67] Li et al [66] Chapman et al [72] Dandala et al [70] Wei et al [62] Li et al [66] Wunnava et al [67] Dandala et al [70] Chapman et al [72] Gligic et al [63] Tao et al [71] Wunnava et al [67] Yang et al [73] Dandala et al [70] Li et al [66] Wei et al [62] Chapman et al [72] Word2vec embeddings self-trained on MIMIC-III [58] (as feature for CRF: GloVe [53] embeddings self-trained on MIMIC-III [58])…”

“…Wei et al [62] Christopoulou et al [74] Chapman et al [72] Dandala et al [70] Wei et al [62] Christopoulou et al [74] Dandala et al [70] Chapman et al [72] Wei et al [62] Christopoulou et al [74] Chapman et al [72] Dandala et al [70] Wei et al [62] Chapman et al [72] Christopoulou et al [74] Dandala et al [70] Wei et al [62] Christopoulou et al [74] Chapman et al [72] Dandala et al [70] Wei et al [62] Christopoulou et al [74] Dandala et al [70] Yang et al [73] Relation type (thus diverging from the most successful architectures for medication NER; see Tables 3 and 4) and rule-based post-processing that outperformed a simple CNN-RNN. Summarizing, the CNN-RNN approach seems more favorable than an (attention-based) BiLSTM, with preferences for self-trained in-domain embeddings.…”