Clinical Dialogue Transcription Error Correction with Self-supervision

Nanayakkara, Gayani; Wiratunga, Nirmalie; Corsar, David; Martin, Kyle; Wijekoon, Anjana

doi:10.1007/978-3-031-47994-6_3

Cited by 1 publication

(1 citation statement)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This resilience is critical for the practical deployment of digital scribes, where they must perform reliably across varying conditions of data quality. [53] The nature of conversations…”

Section: Transcription Qualitymentioning

confidence: 99%

Development and Evaluation of a Digital Scribe: Conversation Summarization Pipeline for Emergency Department Counseling Sessions towards Reducing Documentation Burden

Sezgin,

Sirrianni,

Kranz

2023

Preprint

View full text Add to dashboard Cite

ObjectiveWe present a proof-of-concept digital scribe system as an ED clinical conversation summarization pipeline and report its performance.Materials and MethodsWe use four pre-trained large language models to establish the digital scribe system: T5-small, T5-base, PEGASUS-PubMed, and BART-Large-CNN via zero-shot and fine-tuning approaches. Our dataset includes 100 referral conversations among ED clinicians and medical records. We report the ROUGE-1, ROUGE-2, and ROUGE-L to compare model performance. In addition, we annotated transcriptions to assess the quality of generated summaries.ResultsThe fine-tuned BART-Large-CNN model demonstrates greater performance in summarization tasks with the highest ROUGE scores (F1ROUGE-1=0.49, F1ROUGE-2=0.23, F1ROUGE-L=0.35) scores. In contrast, PEGASUS-PubMed lags notably (F1ROUGE-1=0.28, F1ROUGE-2=0.11, F1ROUGE-L=0.22). BART-Large-CNN’s performance decreases by more than 50% with the zero-shot approach. Annotations show that BART-Large-CNN performs 71.4% recall in identifying key information and a 67.7% accuracy rate.DiscussionThe BART-Large-CNN model demonstrates a high level of understanding of clinical dialogue structure, indicated by its performance with and without fine-tuning. Despite some instances of high recall, there is variability in the model’s performance, particularly in achieving consistent correctness, suggesting room for refinement. The model’s recall ability varies across different information categories.ConclusionThe study provides evidence towards the potential of AI-assisted tools in reducing clinical documentation burden. Future work is suggested on expanding the research scope with larger language models, and comparative analysis to measure documentation efforts and time.

show abstract

“…This resilience is critical for the practical deployment of digital scribes, where they must perform reliably across varying conditions of data quality. [53] The nature of conversations…”

Section: Transcription Qualitymentioning

confidence: 99%