HEAD-QA: A Healthcare Dataset for Complex Reasoning

Vilares, David; Gómez-Rodríguez, Carlos

doi:10.18653/v1/p19-1092

Cited by 26 publications

(24 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These tasks are mostly in multiple-choice forms. In Table 1, we list several representative subjectarea multiple-choice QA datasets: NTCIR-11 QA-Lab (Shibuki et al, 2014), QS (Cheng et al, 2016), MCQA (Guo et al, 2017), ARC (Clark et al, 2018), GeoSQA (Huang et al, 2019b), HEAD-QA (Vilares and Gómez-Rodríguez, 2019), EX-AMS (Hardalov et al, 2020), JEC-QA (Zhong et al, 2020), and MEDQA (Jin et al, 2020). Some multiple-choice MRC datasets for Chinese such as C 3 are collected from language exams designed to test the reading comprehension ability of a human reader.…”

Section: Comparisons With Existing Subject-areamentioning

confidence: 99%

Self-Teaching Machines to Read and Comprehend with Large-Scale Multi-Subject Question-Answering Data

Sun

et al. 2021

Findings of the Association for Computational Linguistics: EMNLP 2021

View full text Add to dashboard Cite

Despite considerable progress, most machine reading comprehension (MRC) tasks still lack sufficient training data to fully exploit powerful deep neural network models with millions of parameters, and it is laborious, expensive, and time-consuming to create largescale, high-quality MRC data through crowdsourcing. This paper focuses on generating more training data for MRC tasks by leveraging existing question-answering (QA) data. We first collect a large-scale multi-subject multiple-choice QA dataset for Chinese, Ex-amQA. We next use incomplete, yet relevant snippets returned by a web search engine as the context for each QA instance to convert it into a weakly-labeled MRC instance. To better use the weakly-labeled data to improve a target MRC task, we evaluate and compare several methods and further propose a self-teaching paradigm. Experimental results show that, upon state-of-the-art MRC baselines, we can obtain +5.1% in accuracy on a multiple-choice Chinese MRC dataset, C 3 , and +3.8% in exact match on an extractive Chinese MRC dataset, CMRC 2018, demonstrating the usefulness of the generated QAbased weakly-labeled data for different types of MRC tasks as well as the effectiveness of self-teaching. ExamQA will be available at https://dataset.org/examqa/.

show abstract

Section: Comparisons With Existing Subject-areamentioning

confidence: 99%

Self-Teaching Machines to Read and Comprehend with Large-Scale Multi-Subject Question-Answering Data

Sun

et al. 2021

Findings of the Association for Computational Linguistics: EMNLP 2021

View full text Add to dashboard Cite

show abstract

“…Medical Question Answering: There have been efforts to create QA datasets for medical reasoning [21,22]. The Medication QA dataset [23] comprises nearly 700 drug-related consumer questions along with information retrieved from reliable websites and scientific papers.…”

Section: Related Workmentioning

confidence: 99%

Q-Pain: A Question Answering Dataset to Measure Social Bias in Pain Management

Logé,

Ross,

Dadey

et al. 2021

View full text Add to dashboard Cite

Recent advances in Natural Language Processing (NLP), and specifically automated Question Answering (QA) systems, have demonstrated both impressive linguistic fluency and a pernicious tendency to reflect social biases. In this study, we introduce Q-Pain, a dataset for assessing bias in medical QA in the context of pain management, one of the most challenging forms of clinical decision-making. Along with the dataset, we propose a new, rigorous framework, including a sample experimental design, to measure the potential biases present when making treatment decisions. We demonstrate its use by assessing two reference Question-Answering systems, GPT-2 and GPT-3, and find statistically significant differences in treatment between intersectional race-gender subgroups, thus reaffirming the risks posed by AI in medical settings, and the need for datasets like ours to ensure safety before medical AI applications are deployed.

show abstract

“…[182] • Multiple Choice Question (MCQ) A few answer options are provided out of which one or more are correct. [188,117].…”

Section: Classification By Answer Typementioning

confidence: 99%

“…• Medical Examination: These datasets are constructed from questions asked in medical certification or other associated exams. These comprise of clinical questions which needs both knowledge and logical reasoning to answer [188].…”

Section: Classification By Sub-domainmentioning

confidence: 99%

Recent Advances in Automated Question Answering In Biomedical Domain

Baksi¹

2021

Preprint

View full text Add to dashboard Cite

The objective of automated Question Answering (QA) systems is to provide answers to user queries in a time efficient manner. The answers are usually found in either databases (or knowledge bases) or a collection of documents commonly referred to as the corpus. In the past few decades there has been a proliferation of acquisition of knowledge and consequently there has been an exponential growth in new scientific articles in the field of biomedicine. Therefore, it has become difficult to keep track of all the information in the domain, even for domain experts. With the improvements in commercial search engines, users can type in their queries and get a small set of documents most relevant for answering their query, as well as relevant snippets from the documents in some cases. However, it may be still tedious and time consuming to manually look for the required information or answers. This has necessitated the development of efficient QA systems which aim to find exact and precise answers to user provided natural language questions in the domain of biomedicine. In this paper, we introduce the basic methodologies used for developing general domain QA systems, followed by a thorough investigation of different aspects of biomedical QA systems, including benchmark datasets and several proposed approaches, both using structured databases and collection of texts. We also explore the limitations of current systems and explore potential avenues for further advancement.

show abstract

HEAD-QA: A Healthcare Dataset for Complex Reasoning

Cited by 26 publications

References 21 publications

Self-Teaching Machines to Read and Comprehend with Large-Scale Multi-Subject Question-Answering Data

Self-Teaching Machines to Read and Comprehend with Large-Scale Multi-Subject Question-Answering Data

Q-Pain: A Question Answering Dataset to Measure Social Bias in Pain Management

Recent Advances in Automated Question Answering In Biomedical Domain

Contact Info

Product

Resources

About