How to Ask Better Questions? A Large-Scale Multi-Domain Dataset for Rewriting Ill-Formed Questions

Chu, Zewei; Chen, Mingda; Jing, Chen; Wang, Miaosen; Gimpel, Kevin; Faruqui, Manaal; Si, Xiance

doi:10.1609/aaai.v34i05.6258

Cited by 11 publications

(7 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Quora dataset contains similar yet differently expressed question pairs from the Quora online Q&A forum. The multi-domain question rewriting (MQR) (Chu et al 2020) dataset consists of ill-formed and well-formed question pairs, for example, "Spaghetti carbonara, mixing" is paired with "How to mix a spaghetti carbonara?". Lastly, we leverage an internal dataset of 300k query logs produced by human agents in the financial QA setting for out-of-sample experiments.…”

Section: Methodology Datasetsmentioning

confidence: 99%

See 1 more Smart Citation

Exploring Fluent Query Reformulations with Text-to-Text Transformers and Reinforcement Learning

Chen,

Yu,

Wang

2020

Preprint

View full text Add to dashboard Cite

Query reformulation aims to alter potentially noisy or ambiguous text sequences into coherent ones closer to natural language questions. In this process, it is also crucial to maintain and even enhance performance in a downstream environments like question answering when rephrased queries are given as input. We explore methods to generate these query reformulations by training reformulators using text-to-text transformers and apply policy-based reinforcement learning algorithms to further encourage reward learning. Query fluency is numerically evaluated by the same class of model fine-tuned on a human-evaluated well-formedness dataset. The reformulator leverages linguistic knowledge obtained from transfer learning and generates more well-formed reformulations than a translation-based model in qualitative and quantitative analysis. During reinforcement learning, it better retains fluency while optimizing the RL objective to acquire question answering rewards and can generalize to out-ofsample textual data in qualitative evaluations. Our RL framework is demonstrated to be flexible, allowing reward signals to be sourced from different downstream environments such as intent classification.

show abstract

Section: Methodology Datasetsmentioning

confidence: 99%

“…Then the MQR dataset is used to further fine-tune the model. The MQR work (Chu et al 2020) suggests that Quora and MQR create a good combination for improving query qualities. This fits our purpose of transforming noisy queries into more fluent reformulations.…”

Section: T5 Frameworkmentioning

confidence: 99%

Exploring Fluent Query Reformulations with Text-to-Text Transformers and Reinforcement Learning

Chen,

Yu,

Wang

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Faruqui and Das annotate the Paralex dataset (Fader et al, 2013) on the well-formedness of the questions. The majority of research efforts have been aimed at reformulating user queries to elicit the best possible answer from the QA system (Yang et al, 2014;Buck et al, 2017;Chu et al, 2019). A complementary line of work uses hate speech detection techniques (Gupta et al, 2020) to filter questions that incite hate on the basis of race, religion, etc.…”

Section: Related Workmentioning

confidence: 99%

Will this Question be Answered? Question Filtering via Answer Model Distillation for Efficient Question Answering

Garg¹,

Moschitti²

2021

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

In this paper we propose a novel approach towards improving the efficiency of Question Answering (QA) systems by filtering out questions that will not be answered by them. This is based on an interesting new finding: the answer confidence scores of state-of-the-art QA systems can be approximated well by models solely using the input question text. This enables preemptive filtering of questions that are not answered by the system due to their answer confidence scores being lower than the system threshold. Specifically, we learn Transformer-based question models by distilling Transformer-based answering models. Our experiments on three popular QA datasets and one industrial QA benchmark demonstrate the ability of our question models to approximate the Precision/Recall curves of the target QA system well. These question models, when used as filters, can effectively trade off lower computation cost of QA systems for lower Recall, e.g., reducing computation by ∼60%, while only losing ∼3−4% of Recall.

show abstract

“…We construct the document-category pair dataset by pairing question titles or descriptions with their corresponding subareas. Question titles, descriptions and subareas are available from Chu et al (2020). Many Stack Exchange subareas have their own corresponding "meta" sites.…”

Section: Natcat Datasetmentioning

confidence: 99%

NatCat: Weakly Supervised Text Classification with Naturally Annotated Resources

Chu¹,

Stratos²,

Gimpel³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

We seek to improve text classification by leveraging naturally annotated data. In particular, we construct a general purpose text categorization dataset (NATCAT) from three online resources: Wikipedia, Reddit, and Stack Exchange. These datasets consist of documentcategory pairs derived from manual curation that occurs naturally by their communities. We build general purpose text classifiers by training on NATCAT and evaluate them on a suite of 11 text classification tasks (CATEVAL). We benchmark different modeling choices and dataset combinations, and show how each task benefits from different NATCAT training resources. 1

show abstract

How to Ask Better Questions? A Large-Scale Multi-Domain Dataset for Rewriting Ill-Formed Questions

Cited by 11 publications

References 17 publications

Exploring Fluent Query Reformulations with Text-to-Text Transformers and Reinforcement Learning

Exploring Fluent Query Reformulations with Text-to-Text Transformers and Reinforcement Learning

Will this Question be Answered? Question Filtering via Answer Model Distillation for Efficient Question Answering

NatCat: Weakly Supervised Text Classification with Naturally Annotated Resources

Contact Info

Product

Resources

About