Improved diagnosis of thyroid cancer aided with deep learning applied to sonographic text reports: a retrospective, multi-cohort, diagnostic study

Abstract

Objective: Large volume radiological text data have been accumulated since the incorporation of electronic health record (EHR) systems in clinical practice. We aimed to determine whether deep natural language processing algorithms could aid radiologists in improving thyroid cancer diagnosis. Methods: Sonographic EHR data were obtained from the EHR database. Pathological reports were used as the gold standard for diagnosing thyroid cancer. We developed thyroid cancer diagnosis based on natural language processing (THCaDxNLP) to interpret unstructured sonographic text reports for thyroid cancer diagnosis. We used the area under the receiver operating characteristic curve (AUROC) as the primary metric to measure the performance of the THCaDxNLP. We compared the performance of thyroid ultrasound radiologists aided with THCaDxNLP vs. those without THCaDxNLP using 5 independent test sets. Results: We obtained a total number of 788,129 sonographic radiological reports. The number of thyroid sonographic data points was 132,277, 18,400 of which were thyroid cancer patients. Among the 5 test sets, the numbers of patients per set were 439, 186, 82, 343, and 171. THCaDxNLP achieved high performance in identifying thyroid cancer patients (the AUROC ranged from 0.857–0.932). Thyroid ultrasound radiologists aided with THCaDxNLP achieved significantly higher performances than those without THCaDxNLP in terms of accuracy (93.8% vs. 87.2%; one-sided t-test, adjusted P = 0.003), precision (92.5% vs. 86.0%; P = 0.018), and F1 metric (94.2% vs. 86.4%; P = 0.007). Conclusions: THCaDxNLP achieved a high AUROC for the identification of thyroid cancer, and improved the accuracy, sensitivity, and precision of thyroid ultrasound radiologists. This warrants further investigation of THCaDxNLP in prospective clinical trials.

Review of Deep Learning Approaches for Thyroid Cancer Diagnosis

“…ere is a significant need for AI-based CAD systems with better designs and practicality that provide consistent nodule management solutions in practice [65,66]. In this paper, we reviewed the recent studies that deployed deep learningbased algorithms for analyzing medical images of thyroid nodules.…”

A Systematic Review of Natural Language Processing Methods and Applications in Thyroidology

“… 12 , 21 Furthermore, some models standardized unstructured data from multiple institutions to generate multicenter data sets. 12 , 19 , 21 , 29 , 34 …”

“…Notably, among the 24 studies identified, only 1 adopted a prospective design, 31 and 2 undertook validation of their NLP models in an external health care setting. 31 , 34 The road to incorporating these NLP interventions into routine research or clinical practice is riddled with several challenges that need careful consideration and concerted efforts to surmount. Specifically in the thyroid field, we hypothesize that the uptake of NLP methods is associated with the complexity of the thyroid-related domains, variations in language expression and reporting styles, completeness and accuracy of clinical documentation (ie, data on patient-specific concerns, complaints, or severity of symptoms depends on the accuracy of providers’ documentation), semantic (ie, misspellings, abbreviations, acronyms, or synonyms), and context (ie, it is challenging to create algorithms that can appropriately extract chronologic descriptions or simultaneous references in situations like a thyroid ultrasound report that includes several nodules), which can affect the NLP outcome, decrease the performance of the algorithm when applied to different institutions, and limit the portability and scalability of the interventions.…”

Transforming Thyroid Cancer Diagnosis and Staging Information from Unstructured Reports to the Observational Medical Outcome Partnership Common Data Model

“…From pathology reports, they focused on the extraction of the nodule size, laterality, histologic type, and a final diagnosis, while we further tried to extract cancer subtype and cancer stage information in this study. Zhang and colleagues 12 developed deep NLP to diagnose thyroid cancer using sonographic text reports. Meanwhile, we focused on regular expression-based NLP of pathology reports and iodine whole-body scan reports for cancer staging.…”