Attention based automated radiology report generation using CNN and LSTM

Sirshar, Mehreen; Paracha, Muhammad Faheem Khalil; Akram, M. Usman; Alghamdi, Norah Saleh; Zaidi, Syeda Zainab Yousuf; Fatima, Tatheer

doi:10.1371/journal.pone.0262209

Cited by 17 publications

(15 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We are also aware of a separate clinical product that also generates an impression section from the physician-dictated findings section (https://www.radai.com accessed on December 1, 2022). Prior studies have used natural language processing and AI to create radiology reports text from images [7,14,15,16]. PSI performed substantially better for findings deemed clinically significant than for incidental findings.…”

Section: Discussionmentioning

confidence: 99%

Artificial intelligence-generated smart impression from 9.8-million radiology reports as training datasets from multiple sites and imaging modalities

Kaviani,

Kalra,

Digumarthy

et al. 2024

Preprint

View full text Add to dashboard Cite

ImportanceAutomatic generation of the impression section of radiology report can help make radiologists efficient and avoid reporting errors.ObjectiveTo evaluate the relationship, content, and accuracy of an Powerscribe Smart Impression (PSI) against the radiologists’ reported findings and impression (RDF).Design, Setting, and ParticipantsThe institutional review board approved retrospective study developed and trained an PSI algorithm (Nuance Communications, Inc.) with 9.8 million radiology reports from multiple sites to generate PSI based on information including the protocol name and the radiologists-dictated findings section of radiology reports. Three radiologists assessed 3879 radiology reports of multiple imaging modalities from 8 US imaging sites. For each report, we assessed if PSI can accurately reproduce the RDF in terms of the number of clinically significant findings and radiologists’ style of reporting while avoiding potential mismatch (with the findings section in terms of size, location, or laterality). Separately we recorded the word count for PSI and RDF. Data were analyzed with Pearson correlation and paired t-tests.Main Outcomes and MeasuresThe data were ground truthed by three radiologists. Each radiologists recorded the frequency of the incidental/significant findings, any inconsistency between the RDF and PSI as well as the stylistic evaluation overall evaluation of PSI. Area under the curve (AUC), correlation coefficient, and the percentages were calculated.ResultsPSI reports were deemed either perfect (91.9%) or acceptable (7.68%) for stylistic concurrence with RDF. Both PSI (mismatched Haller’s Index) and RDF (mismatched nodule size) had one mismatch each. There was no difference between the word counts of PSI (mean 33±23 words/impression) and RDF (mean 35±24 words/impression) (p>0.1). Overall, there was an excellent correlation (r= 0.85) between PSI and RDF for the evolution of findings (negative vs. stable vs. new or increasing vs. resolved or decreasing findings). The PSI outputs (2%) requiring major changes pertained to reports with multiple impression items.Conclusion and RelevanceIn clinical settings of radiology exam interpretation, the Powerscribe Smart Impression assessed in our study can save interpretation time; a comprehensive findings section results in the best PSI output.

show abstract

Section: Discussionmentioning

confidence: 99%

Artificial intelligence-generated smart impression from 9.8-million radiology reports as training datasets from multiple sites and imaging modalities

Kaviani,

Kalra,

Digumarthy

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Impressions were generated by applying cross-attention on findings and input sequence. Reference [29] presented a method where they combined CNN based features with attention layer and LSTM to generate more reliable reports. They utilized IU and MIMIC datasets for evaluation purposes and outperformed simple attention based methods.…”

Section: Medical Report Generationmentioning

confidence: 99%

Vision Transformer and Language Model Based Radiology Report Generation

et al. 2023

Self Cite

View full text Add to dashboard Cite

Recent advancements in transformers exploited computer vision problems which results in state-of-the-art models. Transformer-based models in various sequence prediction tasks such as language translation, sentiment classification, and caption generation have shown remarkable performance. Auto report generation scenarios in medical imaging through caption generation models is one of the applied scenarios for language models and have strong social impact. In these models, convolution neural networks have been used as encoder to gain spatial information and recurrent neural networks are used as decoder to generate caption or medical report. However, using transformer architecture as encoder and decoder in caption or report writing task is still unexplored. In this research, we explored the effect of losing spatial biasness information in encoder by using pre-trained vanilla image transformer architecture and combine it with different pre-trained language transformers as decoder. In order to evaluate the proposed methodology, the Indiana University Chest X-Rays dataset is used where ablation study is also conducted with respect to different evaluations. The comparative analysis shows that the proposed methodology has represented remarkable performance when compared with existing techniques in terms of different performance parameters.

show abstract

“…Nowadays, AI software can automatically generate radiology reports applicable to X-rays [ 49 ]. The automated generation of radiology reports is available for X-rays and has tremendous potential to enhance patients’ clinical diagnosis of diseases.…”

Section: Ai Can Help the Standardization Of Reportmentioning

confidence: 99%

Advancements in Standardizing Radiological Reports: A Comprehensive Review

Pesapane,

Tantrige,

De Marco

et al. 2023

Medicina

View full text Add to dashboard Cite

Standardized radiological reports stimulate debate in the medical imaging field. This review paper explores the advantages and challenges of standardized reporting. Standardized reporting can offer improved clarity and efficiency of communication among radiologists and the multidisciplinary team. However, challenges include limited flexibility, initially increased time and effort, and potential user experience issues. The efforts toward standardization are examined, encompassing the establishment of reporting templates, use of common imaging lexicons, and integration of clinical decision support tools. Recent technological advancements, including multimedia-enhanced reporting and AI-driven solutions, are discussed for their potential to improve the standardization process. Organizations such as the ACR, ESUR, RSNA, and ESR have developed standardized reporting systems, templates, and platforms to promote uniformity and collaboration. However, challenges remain in terms of workflow adjustments, language and format variability, and the need for validation. The review concludes by presenting a set of ten essential rules for creating standardized radiology reports, emphasizing clarity, consistency, and adherence to structured formats.

show abstract

Attention based automated radiology report generation using CNN and LSTM

Cited by 17 publications

References 25 publications

Artificial intelligence-generated smart impression from 9.8-million radiology reports as training datasets from multiple sites and imaging modalities

Artificial intelligence-generated smart impression from 9.8-million radiology reports as training datasets from multiple sites and imaging modalities

Vision Transformer and Language Model Based Radiology Report Generation

Advancements in Standardizing Radiological Reports: A Comprehensive Review

Contact Info

Product

Resources

About