Learning Out-of-Vocabulary Words in Intelligent Personal Agents

Abstract

Semantic parsers play a vital role in intelligent agents to convert natural language instructions to an actionable logical form representation. However, after deployment, these parsers suffer from poor accuracy on encountering out-of-vocabulary (OOV) words, or significant accuracy drop on previously supported instructions after retraining. Achieving both goals simultaneously is non-trivial. In this paper, we propose novel neural networks based parsers to learn OOV words; one incorporating a new hybrid paraphrase generation model, and an enhanced sequence-to-sequence model. Extensive experiments on both benchmark and custom datasets show our new parsers achieve significant accuracy gain on OOV words and phrases, and in the meanwhile learn OOV words while maintaining accuracy on previously supported instructions.

Fast Domain Adaptation of Semantic Parsers via Paraphrase Attention

“…The use of paraphrases to boost performance of semantic parsers have been studied (Berant and Liang, 2014;Ray et al, 2018). Domain adaptation of semantic parsers have been explored in both pre-deployment (Herzig and Berant, 2017;Fan et al, 2017) and postdeployment (Thomason et al, 2015;Azaria et al, 2016;Iyer et al, 2017;Ray et al, 2018) settings, and using both CCG based and neural network parsers. In (Ray et al, 2018), the authors propose new models to effectively learn user specific OOV words by retraining neural semantic parsers.…”

“…The parser P is trained over a labeled training set T. After deployment, users often use their own personal or locale specific vocabulary in queries, some of which are absent in the training vocabulary V. Let p * be a query with OOV words which parser P cannot parse. We follow the post-deployment domain adaptation settings similar to (Azaria et al, 2016;Ray et al, 2018), where using user feedback/dialog, a paraphrased query q * of p * is obtained which is parsable. The main task of domain adaptation is to retrain P using both the given paraphrased sample (p * , q * , l(q * )), and the training set T to obtain an improved personalized parser P .…”

“…Secondly, users from different locale tend to use different vocabulary, and paraphrases making it nearly impossible to collect a comprehensive dataset covering all possible variety of queries. As a result, once deployed, the semantic parsers require frequent retraining for adaptation to the locale and user specific vocabulary (Thomason et al, 2015;Azaria et al, 2016;Ray et al, 2018). Such domain adaptation and personalization is a key feature in current commercial personal agents (Kim et al, 2018).…”

“…First, our parser is trained to use either attention from input query or attention from its paraphrase (referred as paraphrase attention) when available. For learning new vocabulary from paraphrased queries (Azaria et al, 2016;Ray et al, 2018), this naturally enables our parsers to be retrained much faster, since in our parser only the encoder requires retraining. Secondly, by jointly modeling the semantic dependencies between paraphrases, our parser often achieves better accuracy over previous models.…”

A Progressive Model to Enable Continual Learning for Semantic Slot Filling

“…Recently, (Shen et al, 2018a developed cold start algorithms to generate training data with the hope of covering more varieties before deployment. On the other hand, (Ray et al, 2018;Shen et al, 2018b) attempt to personalize the slot filling model. However, they are still restricted to the offline training and cannot be applied to learn new user's expressions after deployment.…”