A text processing pipeline to extract recommendations from radiology reports

Abstract

Communication of follow-up recommendations when abnormalities are identified on imaging studies is prone to error. The absence of an automated system to identify and track radiology recommendations is an important barrier to ensuring timely follow-up of patients especially with non-acute incidental findings on imaging examinations. In this paper, we present a text processing pipeline to automatically identify clinically important recommendation sentences in radiology reports. Our extraction pipeline is based on natural language processing (NLP) and supervised text classification methods. To develop and test the pipeline, we created a corpus of 800 radiology reports double annotated for recommendation sentences by a radiologist and an internist. We ran several experiments to measure the impact of different feature types and the data imbalance between positive and negative recommendation sentences. Our fully statistical approach achieved the best f-score 0.758 in identifying the critical recommendation sentences in radiology reports.

Natural Language Processing Technologies in Radiology Research and Clinical Applications

“…They compared unigrams with and without their part-of-speech assignment and the report sections in which they appeared as features. With use of all features, the system had a sensitivity of 64.6% and an accuracy of 99.7% in 800 reports representing various imaging modalities at one medical center, and the system was only marginally better than use of unigrams alone (62.8% sensitivity, 99.7% accuracy) (24). Importantly, this work assessed how class prevalence in the machine learning training set affected performance.…”

“…Yetisgen-Yildiz et al (24) noted that miscommunication is the second most common cause of radiologist malpractice suits and developed a cascade of linguistic and machine learning methods to determine the likelihood that a sentence contained a recommendation. They compared unigrams with and without their part-of-speech assignment and the report sections in which they appeared as features.…”

“…Although n-grams are powerful features used in many practical systems such as speech recognition, concepts identified by NLP can be more predictive as features, since NLP reduces synonymous findings to standardized names (15). However, this is not always the case because words that are relevant for some characterizations may not have concepts, such as the words "if" and "further," which have been found to be highly predictive for identifying reports with follow-up recommendations (24), thus missing important features.…”

“…Another powerful classification model that has become very popular in recent years is the support vector machine, which implicitly maps the features to a much higher dimensional space so as to derive many complex features automatically from the existing one, giving the model much better adaptivity. Both the maximum entropy and support vector machine models are often encountered in radiology NLP applications (22,24,(26)(27)(28)(29).…”

Unsupervised Topic Modeling in a Large Free Text Radiology Report Repository

“…Natural language processing (NLP) techniques have been previously applied on radiology reports to classify and extract their information contents [3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18]. For instance, the Lexicon Mediated Entropy Reduction (LEXIMER) system extracts and classifies phrases with important findings and recommendations from radiology reports through lexicon-based hierarchical decision trees [4].…”