Inductive Transfer Learning for Detection of Well-Formed Natural Language Search Queries

Syed, Bakhtiyar; Indurthi, Vijayasaradhi; Gupta, Manish; Shrivastava, Manish; Varma, Vasudeva

doi:10.1007/978-3-030-15719-7_6

Cited by 2 publications

(2 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The well-formedness study (Faruqui and Das 2018) focus on binary accuracy using 0.8 as a threshold to determine whether a question is well-formed. Using the same threshold to group the multi-classed predictions, the binary classification accuracy is 79.56%, which is better than 70.7% reported by the best model in the original paper, higher than another transfer learning approach with 75.05% (Syed et al 2019), close to the accuracy of 81.6% from a BERT-based model (Chhina 2020). Accuracy for individual classes are presented in Table 1, we can see that scores 0.0 and 1.0 can be predicted with accuracy, but not for mid-range scores.…”

Section: Well-formednessmentioning

confidence: 68%

“…Their model uses both the question and the context as input, whereas in our approach, only the original noisy query are used as input to the reward-generating black-box QA model at the RL stage. Identifying well-formed questions (Faruqui and Das 2018) by training binary classification models have been studied using BERT (Chhina 2020) and transfer learning with pretrained models (Syed et al 2019). Instead, we investigate more fine-grained 6-way classification using a fined-tuned T5 well-formedness model, leveraged as a proxy for evaluating sequence-level fluency of reformulators.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Exploring Fluent Query Reformulations with Text-to-Text Transformers and Reinforcement Learning

Chen,

Yu,

Wang

2020

Preprint

View full text Add to dashboard Cite

Query reformulation aims to alter potentially noisy or ambiguous text sequences into coherent ones closer to natural language questions. In this process, it is also crucial to maintain and even enhance performance in a downstream environments like question answering when rephrased queries are given as input. We explore methods to generate these query reformulations by training reformulators using text-to-text transformers and apply policy-based reinforcement learning algorithms to further encourage reward learning. Query fluency is numerically evaluated by the same class of model fine-tuned on a human-evaluated well-formedness dataset. The reformulator leverages linguistic knowledge obtained from transfer learning and generates more well-formed reformulations than a translation-based model in qualitative and quantitative analysis. During reinforcement learning, it better retains fluency while optimizing the RL objective to acquire question answering rewards and can generalize to out-ofsample textual data in qualitative evaluations. Our RL framework is demonstrated to be flexible, allowing reward signals to be sourced from different downstream environments such as intent classification.

show abstract

Section: Well-formednessmentioning

confidence: 68%

Section: Related Workmentioning

confidence: 99%