Diagnostic captioning: a survey

Pavlopoulos, John; Kougia, Vasiliki; Androutsopoulos, Ion; Papamichail, Dimitris

doi:10.1007/s10115-022-01684-7

Cited by 12 publications

(2 citation statements)

References 97 publications

(245 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Pathological captioning tasks are being studied recently to automatically generate diagnostic texts based on patient medical images, assist inexperienced doctors, and reduce clinical errors ( 46 ). The typical representative is still PathVQA ( 19 ).…”

Section: Related Studymentioning

confidence: 99%

PathNarratives: Data annotation for pathological human-AI collaborative diagnosis

Zhang

He²,

Wu³

et al. 2023

Front. Med.

View full text Add to dashboard Cite

Pathology is the gold standard of clinical diagnosis. Artificial intelligence (AI) in pathology becomes a new trend, but it is still not widely used due to the lack of necessary explanations for pathologists to understand the rationale. Clinic-compliant explanations besides the diagnostic decision of pathological images are essential for AI model training to provide diagnostic suggestions assisting pathologists practice. In this study, we propose a new annotation form, PathNarratives, that includes a hierarchical decision-to-reason data structure, a narrative annotation process, and a multimodal interactive annotation tool. Following PathNarratives, we recruited 8 pathologist annotators to build a colorectal pathological dataset, CR-PathNarratives, containing 174 whole-slide images (WSIs). We further experiment on the dataset with classification and captioning tasks to explore the clinical scenarios of human-AI-collaborative pathological diagnosis. The classification tasks show that fine-grain prediction enhances the overall classification accuracy from 79.56 to 85.26%. In Human-AI collaboration experience, the trust and confidence scores from 8 pathologists raised from 3.88 to 4.63 with providing more details. Results show that the classification and captioning tasks achieve better results with reason labels, provide explainable clues for doctors to understand and make the final decision and thus can support a better experience of human-AI collaboration in pathological diagnosis. In the future, we plan to optimize the tools for the annotation process, and expand the datasets with more WSIs and covering more pathological domains.

show abstract

Section: Related Studymentioning

confidence: 99%

PathNarratives: Data annotation for pathological human-AI collaborative diagnosis

Zhang

He²,

Wu³

et al. 2023

Front. Med.

View full text Add to dashboard Cite

show abstract

“…A principal abordagem para gerar relatório médico automático a partir de imagem, usando técnicas de Deep Learning, é o Image Captioning, que em sua abordagem mais simples, consiste em uma rede receber a imagem como entrada e produzir uma palavra por vez assim formando o texto do relatório. Esta técnica é amplamente utilizada no contexto de imagens de raio-x de tórax [Pavlopoulos et al 2022]. As arquiteturas, em sua maioria, seguem o modelo Encoder-Decoder, usando Backbones e redes neurais recorrentes (RNN) [Shin et al 2016] , [Zhang et al 2017].…”

Section: Introduc ¸ãOunclassified

Método automático para geração de laudos médicos em imagens de retinografia utilizando Transformer

Dutra,

Lemos,

Almeida

et al. 2024

Anais Do XXIV Simpósio Brasileiro De Computação Aplicada À Saúde (SBCAS 2024)

View full text Add to dashboard Cite

Estima-se que o número de pessoas afetadas por doenças na retina aumentará significativamente nas próximas décadas. O diagnóstico tradicional dessas patologias envolve a análise visual das estruturas da retina, é um processo demorado e requer especialização. Assim, torna-se útil o uso de um sistema automático para suporte ao diagnóstico pelos especialistas. Neste trabalho é apresentado um método automático de geração de relatório médico, usando rede neural convolucional para extração de características da imagem, combinada a uma rede Transformer que sugere o relatório médico inicial. O método proposto apresenta aumento de 30% em BLEU comparado ao melhor método de Image Captioning na base DeepEyeNet, que tem 265 doenças de retina diferentes.

show abstract

ImageCLEF 2021 Best of Labs: The Curious Case of Caption Generation for Medical Images

Nicolson¹,

Dowling²,

Koopman³

2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Diagnostic captioning: a survey

Cited by 12 publications

References 97 publications

PathNarratives: Data annotation for pathological human-AI collaborative diagnosis

PathNarratives: Data annotation for pathological human-AI collaborative diagnosis

Método automático para geração de laudos médicos em imagens de retinografia utilizando Transformer

ImageCLEF 2021 Best of Labs: The Curious Case of Caption Generation for Medical Images

Contact Info

Product

Resources

About