Spatial Semantic-Preserving Latent Space Learning for Accelerated DWI Diagnostic Report Generation

Gasimova, Aydan; Seegoolam, Gavin; Chen, Liang; Bentley, Paul; Rueckert, Daniel

doi:10.1007/978-3-030-59728-3_33

“…Most of the articles of Table 5 were selected due to their high citation count, but two [40,109] were selected due to their claimed state-of-the-art performance. We excluded any articles with fewer than 8 citations, but we note that this also excludes very recent work, as for example the work of [30,66,88,103], who had 1, 0, 4, 0 citations, respectively. The selected articles mainly focus on chest radiographs and only two study other types of images.…”

Section: Literature Search Methodology and Discussionmentioning

confidence: 99%

Diagnostic captioning: a survey

Pavlopoulos

¹

,

Kougia

²

,

Androutsopoulos

³

et al. 2022

View full text Add to dashboard Cite

Diagnostic captioning (DC) concerns the automatic generation of a diagnostic text from a set of medical images of a patient collected during an examination. DC can assist inexperienced physicians, reducing clinical errors. It can also help experienced physicians produce diagnostic reports faster. Following the advances of deep learning, especially in generic image captioning, DC has recently attracted more attention, leading to several systems and datasets. This article is an extensive overview of DC. It presents relevant datasets, evaluation measures, and up-to-date systems. It also highlights shortcomings that hinder DC’s progress and proposes future directions.

show abstract

“…Apart from some familiar topics such as disease detection (Oh et al, 2020;Luo et al, 2020;Lu et al, 2020b;Rajpurkar et al, 2017;Lu et al, 2020a;Ranjan et al, 2018) and lung segmentation (Eslami et al, 2020), the most related computer vision task is the emerging topic of image-based captioning, which aims at generating realistic sentences or topic-related paragraphs to summarize visual contents from images or videos (Vinyals et al, 2015;Xu et al, 2015;Goyal et al, 2017;Rennie et al, 2017;Huang et al, 2019;Feng et al, 2019;Pei et al, 2019;Tran et al, 2020). Not surprisingly, the recent progresses in medical report generation (Jing et al, 2018(Jing et al, , 2019Li et al, 2018Xue et al, 2018;Yuan et al, 2019;Wang et al, 2018;Lovelace and Mortazavi, 2020;Srinivasan et al, 2020;Zhang et al, 2020;Huang et al, 2021;Gasimova et al, 2020;Singh et al, 2019;Nishino et al, 2020) have been particularly influenced by the successes in image-based captioning. The work of (Vinyals et al, 2015;Xu et al, 2015) is among the early approaches in medical report generation, where visual features are extracted by convolution neural networks (CNNs); they are subsequently fed into recurrent neural networks (RNNs) to generate textual descriptions.…”

Section: Image-based Captioning and Medical Report Generationmentioning

confidence: 99%

Automated Generation of Accurate & Fluent Medical X-ray Reports

Nguyen¹,

Nie²,

Badamdorj³

et al. 2021

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Our paper aims to automate the generation of medical reports from chest X-ray image inputs, a critical yet time-consuming task for radiologists. Existing medical report generation efforts emphasize producing human-readable reports, yet the generated text may not be well aligned to the clinical facts. Our generated medical reports, on the other hand, are fluent and, more importantly, clinically accurate. This is achieved by our fully differentiable and end-to-end paradigm that contains three complementary modules: taking the chest X-ray images and clinical history document of patients as inputs, our classification module produces an internal checklist of disease-related topics, referred to as enriched disease embedding; the embedding representation is then passed to our transformer-based generator, to produce the medical report; meanwhile, our generator also creates a weighted embedding representation, which is fed to our interpreter to ensure consistency with respect to diseaserelated topics. Empirical evaluations demonstrate very promising results achieved by our approach on commonly-used metrics concerning language fluency and clinical accuracy. Moreover, noticeable performance gains are consistently observed when additional input information is available, such as the clinical document and extra scans from different views.* indicates equal contribution.

show abstract

“…Apart from some familiar topics such as disease detection (Oh et al, 2020;Luo et al, 2020;Lu et al, 2020b;Rajpurkar et al, 2017;Lu et al, 2020a;Ranjan et al, 2018) and lung segmentation (Eslami et al, 2020), the most related computer vision task is the emerging topic of image-based captioning, which aims at generating realistic sentences or topic-related paragraphs to summarize visual contents from images or videos (Vinyals et al, 2015;Xu et al, 2015;Goyal et al, 2017;Rennie et al, 2017;Huang et al, 2019;Feng et al, 2019;Pei et al, 2019;Tran et al, 2020). Not surprisingly, the recent progresses in medical report generation (Jing et al, 2018(Jing et al, , 2019Li et al, 2018Xue et al, 2018;Yuan et al, 2019;Wang et al, 2018;Lovelace and Mortazavi, 2020;Srinivasan et al, 2020;Zhang et al, 2020;Huang et al, 2021;Gasimova et al, 2020;Singh et al, 2019;Nishino et al, 2020) have been particularly influenced by the successes in image-based captioning.…”

Section: Image-based Captioning and Medical Report Generationmentioning

confidence: 99%

Automated Generation of Accurate \& Fluent Medical X-ray Reports

Nguyen¹,

Nie²,

Badamdorj³

et al. 2021

Preprint

1

0

View full text Add to dashboard Cite

Our paper focuses on automating the generation of medical reports from chest X-ray image inputs, a critical yet time-consuming task for radiologists. Unlike existing medical report generation efforts that tend to produce human-readable reports, we aim to generate medical reports that are both fluent and clinically accurate. This is achieved by our fully differentiable and end-to-end paradigm containing three complementary modules: taking the chest X-ray images and clinical history document of patients as inputs, our classification module produces an internal checklist of disease-related topics, referred to as enriched disease embedding; the embedding representation is then passed to our transformerbased generator, giving rise to the medical reports; meanwhile, our generator also produces the weighted embedding representation, which is fed to our interpreter to ensure consistency with respect to disease-related topics. Our approach achieved promising results on commonly-used metrics concerning language fluency and clinical accuracy. Moreover, noticeable performance gains are consistently observed when additional input information is available, such as the clinical document and extra scans of different views.

show abstract

Spatial Semantic-Preserving Latent Space Learning for Accelerated DWI Diagnostic Report Generation

Cited by 4 publications

References 24 publications

Diagnostic captioning: a survey

Diagnostic captioning: a survey

Automated Generation of Accurate & Fluent Medical X-ray Reports

Automated Generation of Accurate \& Fluent Medical X-ray Reports

Contact Info

Product

Resources

About