COSMO: Conditional SEQ2SEQ-based Mixture Model for Zero-Shot Commonsense Question Answering

Abstract

Commonsense reasoning refers to the ability of evaluating a social situation and acting accordingly. Identification of the implicit causes and effects of a social context is the driving capability which can enable machines to perform commonsense reasoning. The dynamic world of social interactions requires context-dependent on-demand systems to infer such underlying information. However, current approaches in this realm lack the ability to perform commonsense reasoning upon facing an unseen situation, mostly due to incapability of identifying a diverse range of implicit social relations. Hence they fail to estimate the correct reasoning path. In this paper, we present Conditional SEQ2SEQ-based Mixture model (COSMO), which provides us with the capabilities of dynamic and diverse content generation. We use COSMO to generate context-dependent clauses, which form a dynamic Knowledge Graph (KG) on-the-fly for commonsense reasoning. To show the adaptability of our model to context-dependant knowledge generation, we address the task of zero-shot commonsense question answering. The empirical results indicate an improvement of up to +5.2% over the state-of-the-art models.

Paraphrasing Techniques for Maritime QA system

“…Specifically, by designing and constructing an appropriate input sequence of words (called a prompt), one is able to induce the model to produce the desired output sequence (i.e., a paraphrased utterance in this context)without changing its parameters through fine-tuning at all. In particular, in the low-data regime, empirical analysis shows that, either for manually picking hand-crafted prompts [21] or automatically building auto-generated prompts [8], [16] taking prompts for tuning models is surprisingly effective for the knowledge stimulation and model adaptation of pre-trained language model. However, none of these methods reports the use of a few-shot pre-trained language model to directly generate few-shot and zero-shot paraphrased utterances.…”

“…3) Paraphrasing using fine-tuned language model generation: Previous researches have proposed several pretrained autoregressive language models, such as GPT-2 [28] and ProphetNet [26]. Other studies such as [21] use these language models to solve various downstream tasks and achieve SOTA performance. In this work, following [41], we fine-tune an autoregressive language model BART [15] on the dataset from [14], which is a subset of the PARANMT corpus [38], to generate syntactically and lexically diversified paraphrases.…”