Evaluating automatic methods to extract patients' supplement use from clinical reports

Abstract

The widespread prevalence of dietary supplements has drawn extensive attention due to the safety and efficacy issue. Clinical notes document a great amount of detailed information on dietary supplement usage, thus providing a rich source for clinical research on supplement safety surveillance. Identification the use status of dietary supplements is one of the initial steps for the ultimate goal of the supplement safety surveillance. In this study, we built rule-based and machine learning-based classifiers to automatically classify the use status of supplements into four categories: Continuing (C), Discontinued (D), Started (S), and Unclassified (U). In comparison to the machine learning classifier trained on the same datasets, the rule-based classifier showed a better performance with F-measure in the C, D, S, U status of 0.93, 0.98, 0.95, and 0.83, respectively. We further analyzed the errors generated by the rule-based classifier. The classifier can be potentially applied to extract supplement information from clinical notes for supporting research and clinical practice related to patient safety on supplement usage.

Using natural language processing methods to classify use status of dietary supplements in clinical notes

“…Like the previous study [ 13 ], the sources of errors were mainly made up of three parts. First, there are new patterns we failed to generate from the training set.…”

“…The rule-based classifier was developed on the training data and further tested using the test data. Two of our previous studies [ 12 , 13 ] have shown that indicator words are extremely important in recognition and detection use status of dietary supplements. Based on the training data, a set of rules were generated using a variety of status indicators, which were compiled from reviewing the clinical notes and incorporated from other works identifying the use status of medications.…”

“…Compared with the previous study [ 13 ], more feature sets were trained when building the machine learning-based classifier, such as bigrams, the combination of unigrams, bigrams, and trigrams. Totally, five classification algorithms were trained along with nine types of feature sets on the training data.…”

“…Clinical notes have been extensively investigated to detect and recognize the medication use status. As for dietary supplements, we have previously investigated the detection of use status from clinical notes using both rule-based [ 11 ] and machine learning-based methods [ 12 ], and we also compared the performance of both classifiers based on a corpus with a smaller sample size of 1300 sentences [ 13 ].…”

“…In this study, we doubled the size of the corpus (2500 sentences) compared to our previous studies [ 13 ] and compared the performances. We tested more feature sets (e.g., bigrams, TF-IDF) with the supervised machine learning classification algorithms.…”