Using Paraphrasing and Memory-Augmented Models to Combat Data Sparsity in Question Interpretation with a Virtual Patient Dialogue System

Abstract

When interpreting questions in a virtual patient dialogue system, one must inevitably tackle the challenge of a long tail of relatively infrequently asked questions. To make progress on this challenge, we investigate the use of paraphrasing for data augmentation and neural memory-based classification, finding that the two methods work best in combination. In particular, we find that the neural memory-based approach not only outperforms a straight CNN classifier on low frequency questions, but also takes better advantage of the augmented data created by paraphrasing, together yielding a nearly 10% absolute improvement in accuracy on the least frequently asked questions.

Curio SmartChat : A system for Natural Language Question Answering for Self-Paced K-12 Learning

“…Increasing numbers of students now spend class time working one-on-one with adaptive learning platforms (Baker, 2016), and in these contexts, multiple students may have questions at the same time, and teachers may not be able to answer all questions at the same time (Schofield, 1995). This challenge has led to the idea of automated question answering systems in education (Louwerse et al, 2002;Corbett et al, 2005;Milik et al, 2006;Jin et al, 2018), where students can ask questions in natural language. Different than simply a search engine, educational question answering systems attempt to provide answers focused on current content, set at an appropriate level for the student's current stage of learning.…”

Handling Class Imbalance in Low-Resource Dialogue Systems by Combining Few-Shot Classification and Interpolation

“…At test time, an utternace x test is encoded to obtain r test = f (x test ; θ). 3 For each class, we perform a 1-nearest-neighbor search 4 on the training set using r test and set the corresponding elements of the class score c nn test ∈ R C to be the inverse distance to r test . We also compute the classifier class scores on the unmixed test utterance, c class test = g(r test ; φ).…”

“…Various methods have been proposed to handling rare classes in this low-resource dataset, including memory and paraphrasing [4], text-to-phonetic data-augmentation [5] and an ensemble of rule-based and deep learning based models [6]. Recently, self-attention has shown to work particularly well for rare class classification [7].…”

“…Further, pairs of training instances are interpolated using mixup [10] for augmenting the data. A classifier trained with augmented data helps to maintain performance of the model on the frequent classes, unlike other pairwise learning frameworks [4] that require additional model ensembling to maintain overall performance.…”

Equipping Educational Applications with Domain Knowledge

“…Educational applications tend to target a specific subject, in other words, a specific domain, such as the medical domain in the case of (Jin et al, 2018). Thus, building these applications with underlying NLP algorithms, would typically require a large domain-specific corpus.…”