CliCR: a Dataset of Clinical Case Reports for Machine Reading Comprehension

Abstract

We present a new dataset for machine comprehension in the medical domain. Our dataset uses clinical case reports with around 100,000 gap-filling queries about these cases. We apply several baselines and state-of-the-art neural readers to the dataset, and observe a considerable gap in performance (20% F1) between the best human and machine readers. We analyze the skills required for successful answering and show how reader performance varies depending on the applicable skills. We find that inferences using domain knowledge and object tracking are the most frequently required skills, and that recognizing omitted information and spatio-temporal reasoning are the most difficult for the machines. * We provide the information about accessing the dataset, as well as the code for the experiments, at http://github. com/clips/clicr.

A Gated Dilated Convolution with Attention Model for Clinical Cloze-Style Reading Comprehension

“…The performances of the baselines rand-entity and maxfreq-entity presented in [7] are very poor because a random entity and the most frequent entity in the passage are used as answers, respectively. The lang-model method performs poor because it is based on queries only, without reading the document, it is difficult to provide accurate answers.…”

“…These tasks have attracted some researchers to carry out various researches, and have played dramatic roles in promoting researches in the clinical medical field [6]. And some related data sets have been proposed, such as CliCR [7], PubMedQA [8], Chimed [2] and emrQA [3] etc. Besides, the clinical field has accumulated extensive experience and knowledge, some of which have been uploaded to PubMed, one of the literature databases in the biomedical field, and has nearly 2 million publications with case types [9,10].…”

Neural Machine Reading Comprehension: Methods and Trends

“…As questions are not proposed directly from documents, this task is challenging and some information extraction methods fail to deal with it. This methodology of creating MRC datasets enlightens lots of other researches [77,52,69]. In order to avoid that questions can be answered by knowledge out of the documents, all entities in documents are anonymized by random markers.…”

“…-CliCR To address the problem that there are scarce datasets for specific domains, Suster et al [77] build a large-scale cloze-style dataset based on clinical case reports for healthcare and medicine. Similar to the CNN & Daily Mail, summary points of each case reports are used to create queries by blanking out a medical entity.…”

Machine Comprehension Improves Domain-Specific Japanese Predicate-Argument Structure Analysis

“…• We improve Japanese PAS analysis by combining the PAS-QA and RC-QA datasets. (Welbl et al, 2017;Suster and Daelemans, 2018;Pampari et al, 2018).…”