Implementation and comparison of two text mining methods with a standard pharmacovigilance method for signal detection of medication errors

Abstract

Background: Medication errors have been identified as the most common preventable cause of adverse events. The lack of granularity in medication error terminology has led pharmacovigilance experts to rely on information in individual case safety reports' (ICSRs) codes and narratives for signal detection, which is both time consuming and labour intensive. Thus, there is a need for complementary methods for the detection of medication errors from ICSRs. The aim of this study is to evaluate the utility of two natural language processing text mining methods as complementary tools to the traditional approach followed by pharmacovigilance experts for medication error signal detection. Methods: The safety surveillance advisor (SSA) method, I2E text mining and University of Copenhagen Center for Protein Research (CPR) text mining, were evaluated for their ability to extract cases containing a type of medication error where patients extracted insulin from a prefilled pen or cartridge by a syringe. A total of 154,209 ICSRs were retrieved from Novo Nordisk's safety database from January 1987 to February 2018. Each method was evaluated by recall (sensitivity) and precision (positive predictive value). Results: We manually annotated 2533 ICSRs to investigate whether these contained the sought medication error. All these ICSRs were then analysed using the three methods. The recall was 90.4, 88.1 and 78.5% for the CPR text mining, the SSA method and the I2E text mining, respectively. Precision was low for all three methods ranging from 3.4% for the SSA method to 1.9 and 1.6% for the CPR and I2E text mining methods, respectively.

Healthcare professionals’ perceptions on medication communication challenges and solutions – text mining and manual content analysis - cross-sectional study

“…The availability of digital free-text data on medication incidents is increasing, and the analysis of such data puts a time strain on research and clinical analysis [ 27 ]. New, rapid analysis methods utilizing artificial intelligence are necessary.…”

“…The NLP method can be divided into information retrieval, whereby classified data is retrieved, and information extraction, which refers to the extraction of non-classified information from free text [ 29 ]. NLP approaches can be divided into supervised [ 27 , 29 ] and unsupervised text-mining methods [ 29 , 30 ]. The use of supervised mining for the detection of medication incidents is currently increasing [ 31 ].…”

“…This method aims to extract information from free-text data; when presented with a “bag of words” [ 33 ], this content is mined for lists of frequent words, related words or themes, and bunches of words that form text topics or clusters [ 34 ]. Text mining is time effective in the analysis of large data sets concerning medication incidents [ 27 , 30 , 31 ], but it is seldom applied to questionnaire-sized, free-text samples [ 35 ]. Only a few studies have compared text mining and manual text analysis [ 27 ].…”

“…Text mining is time effective in the analysis of large data sets concerning medication incidents [ 27 , 30 , 31 ], but it is seldom applied to questionnaire-sized, free-text samples [ 35 ]. Only a few studies have compared text mining and manual text analysis [ 27 ]. To our knowledge, evidence on the efficacy of the text-mining method for studying medication communication challenges is lacking.…”

Year 2020 (with COVID): Observation of Scientific Literature on Clinical Natural Language Processing

“…Indeed, there is a significant number of existing corpora, datasets and resources available in English. Yet, we observe an increasing number of publications dedicated to other languages and a greater variety of languages: Arabic [ 20 ], Chinese [ 21 22 23 24 25 26 ], Croatian [ 27 ], Finnish [ 28 , 29 ], French [ 30 , 31 ], German [ 32 33 34 ], Hebrew [ 35 ], Italian [ 36 37 38 ], Japanese [ 39 , 40 ], Korean [ 41 , 42 ], Norwegian [ 43 ], Portuguese [ 44 ], Spanish [ 45 46 47 48 ], Swedish [ 49 ], and Turkish [ 28 ]. Overall, we believe that the trend observed in previous years is continuing.…”