Towards data-driven medical imaging using natural language processing in patients with suspected urolithiasis

Jungmann, Florian; Kämpgen, Benedikt; Mildenberger, P; Tsaur, Igor; Jorg, Tobias; Düber, Christoph; Mildenberger, P.; Kloeckner, Roman

doi:10.1016/j.ijmedinf.2020.104106

Cited by 17 publications

(16 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Automated processing of text is the domain of natural language processing (NLP) and has an increasing role in healthcare [13]. NLP has been applied in various applications in radiology to annotate texts or extract information [14][15][16]. Natural language processing has evolved from handcrafted rule-based algorithms to machine learning-based approaches and deep learning-based methods [17][18][19][20][21][22][23][24].…”

Section: Introductionmentioning

confidence: 99%

Deep Learning-Based Natural Language Processing in Radiology: The Impact of Report Complexity, Disease Prevalence, Dataset Size, and Algorithm Type on Model Performance

2021

View full text Add to dashboard Cite

In radiology, natural language processing (NLP) allows the extraction of valuable information from radiology reports. It can be used for various downstream tasks such as quality improvement, epidemiological research, and monitoring guideline adherence. Class imbalance, variation in dataset size, variation in report complexity, and algorithm type all influence NLP performance but have not yet been systematically and interrelatedly evaluated. In this study, we investigate these factors on the performance of four types [a fully connected neural network (Dense), a long short-term memory recurrent neural network (LSTM), a convolutional neural network (CNN), and a Bidirectional Encoder Representations from Transformers (BERT)] of deep learning-based NLP. Two datasets consisting of radiologist-annotated reports of both trauma radiographs (n = 2469) and chest radiographs and computer tomography (CT) studies (n = 2255) were split into training sets (80%) and testing sets (20%). The training data was used as a source to train all four model types in 84 experiments (Fracture-data) and 45 experiments (Chest-data) with variation in size and prevalence. The performance was evaluated on sensitivity, specificity, positive predictive value, negative predictive value, area under the curve, and F score. After the NLP of radiology reports, all four model-architectures demonstrated high performance with metrics up to > 0.90. CNN, LSTM, and Dense were outperformed by the BERT algorithm because of its stable results despite variation in training size and prevalence. Awareness of variation in prevalence is warranted because it impacts sensitivity and specificity in opposite directions.

show abstract

Section: Introductionmentioning

confidence: 99%

Deep Learning-Based Natural Language Processing in Radiology: The Impact of Report Complexity, Disease Prevalence, Dataset Size, and Algorithm Type on Model Performance

2021

View full text Add to dashboard Cite

show abstract

“…We used a commercially available NLP pipeline that implements a common approach 8,51 comprised of cleansing, contextualization and concept recognition as well as negation detection trained explicitly for German and English RadLex mappings 1,43 . This fully automated approach to generate bag-of-RadLex mappings is advantageous compared to standard BOW 35 approaches, as it already captures domain-specific knowledge including negation and affirmation 3 .…”

Section: Discussionmentioning

confidence: 99%

“…were mapped to English RadLex terms using a proprietary NLP tool, the Healthcare Analytics Services (HAS) by Empolis Information Management GmbH (Kaiserslautern, Germany; https ://www.empol is.com/en/). As previously described 1,43 , HAS implements a common NLP pipeline consisting of cleansing (e.g., replacement of abbreviations), contextualization (e.g. into segments "clinical information", "findings", and "conclusion"), concept recognition using RadLex, and negation detection ("affirmed", "negated", and "speculated") 77 .…”

Section: Methodsmentioning

confidence: 99%

“…into segments "clinical information", "findings", and "conclusion"), concept recognition using RadLex, and negation detection ("affirmed", "negated", and "speculated") 77 . HAS was pre-trained on ~ 45 k German radiological reports 1,43 . For concept recognition, a full text index and morphosyntactic operations such as tokenization, lemmatization, part of speech tagging, decompounding, noun phrase extraction and sentence detection were used.…”

Section: Methodsmentioning

confidence: 99%

“…ELNET showed a similar performance profile to RFs when fitted on the 28 HEAF but it achieved a narrower fivefold CV confidence range of its accuracies (78-86%) while obtaining similar AUC, BS and LL scores ( Table 1). The mixing parameter alpha (α) was chosen 3 out of 5 times to fit ridge (0) or ridge-like (0.1, 0.1) models while twice to fit lasso (1) or lasso-like (0.8) models on the outer folds.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Comparative analysis of machine learning algorithms for computer-assisted reporting based on fully automated cross-lingual RadLex mappings

Maros

Cho

Junge

et al. 2021

Sci Rep

Self Cite

View full text Add to dashboard Cite

Computer-assisted reporting (CAR) tools were suggested to improve radiology report quality by context-sensitively recommending key imaging biomarkers. However, studies evaluating machine learning (ML) algorithms on cross-lingual ontological (RadLex) mappings for developing embedded CAR algorithms are lacking. Therefore, we compared ML algorithms developed on human expert-annotated features against those developed on fully automated cross-lingual (German to English) RadLex mappings using 206 CT reports of suspected stroke. Target label was whether the Alberta Stroke Programme Early CT Score (ASPECTS) should have been provided (yes/no:154/52). We focused on probabilistic outputs of ML-algorithms including tree-based methods, elastic net, support vector machines (SVMs) and fastText (linear classifier), which were evaluated in the same 5 × fivefold nested cross-validation framework. This allowed for model stacking and classifier rankings. Performance was evaluated using calibration metrics (AUC, brier score, log loss) and -plots. Contextual ML-based assistance recommending ASPECTS was feasible. SVMs showed the highest accuracies both on human-extracted- (87%) and RadLex features (findings:82.5%; impressions:85.4%). FastText achieved the highest accuracy (89.3%) and AUC (92%) on impressions. Boosted trees fitted on findings had the best calibration profile. Our approach provides guidance for choosing ML classifiers for CAR tools in fully automated and language-agnostic fashion using bag-of-RadLex terms on limited expert-labelled training data.

show abstract

Natural language processing of radiology reports to investigate the effects of the COVID-19 pandemic on the incidence and age distribution of fractures

et al. 2021

Self Cite

View full text Add to dashboard Cite

Objective During the COVID-19 pandemic, the number of patients presenting in hospitals because of emergency conditions decreased. Radiology is thus confronted with the effects of the pandemic. The aim of this study was to use natural language processing (NLP) to automatically analyze the number and distribution of fractures during the pandemic and in the 5 years before the pandemic. Materials and methods We used a pre-trained commercially available NLP engine to automatically categorize 5397 radiological reports of radiographs (hand/wrist, elbow, shoulder, ankle, knee, pelvis/hip) within a 6-week period from March to April in 2015–2020 into “fracture affirmed” or “fracture not affirmed.” The NLP engine achieved an F1 score of 0.81 compared to human annotators. Results In 2020, we found a significant decrease of fractures in general (p < 0.001); the average number of fractures in 2015–2019 was 295, whereas it was 233 in 2020. In children and adolescents (p < 0.001), and in adults up to 65 years (p = 0.006), significantly fewer fractures were reported in 2020. The number of fractures in the elderly did not change (p = 0.15). The number of hand/wrist fractures (p < 0.001) and fractures of the elbow (p < 0.001) was significantly lower in 2020 compared with the average in the years 2015–2019. Conclusion NLP can be used to identify relevant changes in the number of pathologies as shown here for the use case fracture detection. This may trigger root cause analysis and enable automated real-time monitoring in radiology.

show abstract

Towards data-driven medical imaging using natural language processing in patients with suspected urolithiasis

Cited by 17 publications

References 22 publications

Deep Learning-Based Natural Language Processing in Radiology: The Impact of Report Complexity, Disease Prevalence, Dataset Size, and Algorithm Type on Model Performance

Deep Learning-Based Natural Language Processing in Radiology: The Impact of Report Complexity, Disease Prevalence, Dataset Size, and Algorithm Type on Model Performance

Comparative analysis of machine learning algorithms for computer-assisted reporting based on fully automated cross-lingual RadLex mappings

Natural language processing of radiology reports to investigate the effects of the COVID-19 pandemic on the incidence and age distribution of fractures

Contact Info

Product

Resources

About