Crowdsourcing Question-Answer Meaning Representations

Michael, Julian; Stanovsky, Gabriel; He, Luheng; Dagan, Ido; Zettlemoyer, Luke

doi:10.18653/v1/n18-2089

Cited by 51 publications

(39 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Question-Answer Meaning Representation, or QAMR (Michael et al, 2018), was recently proposed as an extension of QA-SRL. Like QA-SRL, QAMR represents predicate-argument structure with a set of question-answer pairs about a sentence, where each answer is a span from the sentence.…”

Section: The Qamr Corpusmentioning

confidence: 99%

See 1 more Smart Citation

Supervised Open Information Extraction

Stanovsky

Michael

Zettlemoyer

et al. 2018

Proceedings of the 2018 Conference of the North American Chapter Of the Association for Computational Linguistics: Hu

Self Cite

227

268

View full text Add to dashboard Cite

We present data and methods that enable a supervised learning approach to Open Information Extraction (Open IE). Central to the approach is a novel formulation of Open IE as a sequence tagging problem, addressing challenges such as encoding multiple extractions for a predicate. We also develop a bi-LSTM transducer, extending recent deep Semantic Role Labeling models to extract Open IE tuples and provide confidence scores for tuning their precision-recall tradeoff. Furthermore, we show that the recently released Question-Answer Meaning Representation dataset can be automatically converted into an Open IE corpus which significantly increases the amount of available training data. Our supervised model, made publicly available, 1 outperforms the state-of-the-art in Open IE on benchmark datasets.

show abstract

Section: The Qamr Corpusmentioning

confidence: 99%

“…As described by Michael et al (2018), QAMR annotations were gathered via crowdsourcing in a two-stage pipeline over Wikipedia and Wikinews text. We use the training partition of the QAMR dataset, which consists of 51,063 QA pairs over 3,938 sentences.…”

Section: The All Words Open Ie Corpusmentioning

confidence: 99%

Supervised Open Information Extraction

Stanovsky

Michael

Zettlemoyer

et al. 2018

Proceedings of the 2018 Conference of the North American Chapter Of the Association for Computational Linguistics: Hu

Self Cite

227

268

View full text Add to dashboard Cite

show abstract

“…We developed a taxonomy of nine reasons for why answers may differ, which are summarized in Table 1. Six of the nine reasons are inspired by the crowdsourcing literature -INV [32], DFF [44], AMB [24,26,43], SBJ [33,43,9], SYN [32], and SPM [41,42,14,15]. Two of the reasons are inspired by prior visual question answering work [20] -LQI and IVE.…”

Section: Labeled Datasetsmentioning

confidence: 99%

Why Does a Visual Question Have Different Answers?

Bhattacharya

Gurari

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Visual question answering is the task of returning the answer to a question about an image. A challenge is that different people often provide different answers to the same visual question. To our knowledge, this is the first work that aims to understand why. We propose a taxonomy of nine plausible reasons, and create two labelled datasets consisting of ∼45,000 visual questions indicating which reasons led to answer differences. We then propose a novel problem of predicting directly from a visual question which reasons will cause answer differences as well as a novel algorithm for this purpose. Experiments demonstrate the advantage of our approach over several related baselines on two diverse datasets. We publicly share the datasets and code at https://vizwiz.org.

show abstract

“…It is very likely that some complex annotation tasks require people to read dozens of pages of annotation guidelines, but once decomposed into smaller subtasks, even laymen can handle them. Annotation schemes driven by crowdsourced questionanswering, known to provide only partial coverage are successful examples of this idea (He et al, 2015;Michael et al, 2017). Therefore, this paper is hopefully interesting to a broad audience.…”

Section: Dicussion and Conclusionmentioning

confidence: 99%

Partial Or Complete, That’s The Question

Ning

Fan

et al. 2019

Proceedings of the 2019 Conference of the North

View full text Add to dashboard Cite

For many structured learning tasks, the data annotation process is complex and costly. Existing annotation schemes usually aim at acquiring completely annotated structures, under the common perception that partial structures are of low quality and could hurt the learning process. This paper questions this common perception, motivated by the fact that structures consist of interdependent sets of variables. Thus, given a fixed budget, partly annotating each structure may provide the same level of supervision, while allowing for more structures to be annotated. We provide an information theoretic formulation for this perspective and use it, in the context of three diverse structured learning tasks, to show that learning from partial structures can sometimes outperform learning from complete ones. Our findings may provide important insights into structured data annotation schemes and could support progress in learning protocols for structured tasks.

show abstract

Crowdsourcing Question-Answer Meaning Representations

Cited by 51 publications

References 22 publications

Supervised Open Information Extraction

Supervised Open Information Extraction

Why Does a Visual Question Have Different Answers?

Partial Or Complete, That’s The Question

Contact Info

Product

Resources

About