MedFilter: Improving Extraction of Task-relevant Utterances through Integration of Discourse Structure and Ontological Knowledge

Abstract

Information extraction from conversational data is particularly challenging because the task-centric nature of conversation allows for effective communication of implicit information by humans, but is challenging for machines. The challenges may differ between utterances depending on the role of the speaker within the conversation, especially when relevant expertise is distributed asymmetrically across roles. Further, the challenges may also increase over the conversation as more shared context is built up through information communicated implicitly earlier in the dialogue. In this paper, we propose the novel modeling approach MEDFILTER, which addresses these insights in order to increase performance at identifying and categorizing task-relevant utterances, and in so doing, positively impacts performance at a downstream information extraction task. We evaluate this approach on a corpus of nearly 7,000 doctor-patient conversations where MEDFILTER is used to identify medically relevant contributions to the discussion (achieving a 10% improvement over SOTA baselines in terms of area under the PR curve). Identifying task-relevant utterances benefits downstream medical processing, achieving improvements of 15%, 105%, and 23% respectively for the extraction of symptoms, medications, and complaints.

The digital scribe in clinical practice: a scoping review and research agenda

“…After screening the titles and abstracts of these articles, we assessed 144 full-text articles for eligibility. We included 20 articles [19][20][21][22][23][24][25][26][27][28][29][30][31][32][33][34][35][36][37][38] for our analysis (Fig. 1 and Supplementary Table 2).…”

“…1 and Supplementary Table 2). Of these, ten were conference proceedings [19][20][21]23,27,28,32,38 , seven were workshop proceedings 22,26,29,[34][35][36][37] , two were journal articles 24,25 , and three were Arxiv preprints 30,31,33 .…”

“…Three studies focused on improving the ASR for clinical conversations as the first step towards accurately extracting information from them 19,21,36 . Eleven studies chose to manually transcribe the conversations and performed NLP tasks on the transcripts 20,22,24,25,27,[30][31][32]34,35,40 . Five studies used input data representative of the input of an implemented digital scribe (ASR transcripts or chat dialogs) 26,28,33,37,38 .…”

“…Settings differed greatly between studies, as most did not define a specific specialty 19,[21][22][23]26,[31][32][33][34][35][36]38 , while others were focused on primary care 20,25,27 , home hemodialysis 24 , orthopedic encounters 37 , cardiology, family medicine, internal medicine 31 , and patient-clinician dialogs via a telemedicine platform 28 . Fifteen studies were performed by or in collaboration with a company [19][20][21]23,[25][26][27][28]30,[33][34][35][36][37] .…”

“…The NLP tasks that were performed could be split into three categories: entity extraction 20,[25][26][27]30,32,35,38 , classification 22,24,[30][31][32][33][34][35] , and summarization 22,24,28,29,31,37 (see Fig. 2 and Supplementary Table 4).…”