CosMo: Conditional Seq2Seq-based Mixture Model for Zero-Shot Commonsense Question Answering

Moghimifar, Farhad; Qu, Lizhen; Yang, Zhuo; Baktashmotlagh, Mahsa; Haffari, Gholamreza

doi:10.18653/v1/2020.coling-main.467

Cited by 5 publications

(5 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent approaches have tried to develop models with the human psychological and behavioural capabilities (Jiang et al, 2021;Botzer et al, 2022;Lourie et al, 2021). Other approaches targeted identifying implicit social paradigms by developing sequence generation models (Moghimifar et al, 2020;Bosselut et al, 2019). However, the task of socio-cultural norm discovery has been overlooked, mostly due to the lack of proper annotated data (Fung et al, 2022).…”

Section: Related Workmentioning

confidence: 99%

NormMark: A Weakly Supervised Markov Model for Socio-cultural Norm Discovery

Moghimifar¹,

Qu²,

Wu³

et al. 2023

Findings of the Association for Computational Linguistics: ACL 2023

View full text Add to dashboard Cite

Norms, which are culturally accepted guidelines for behaviours, can be integrated into conversational models to generate utterances that are appropriate for the socio-cultural context. Existing methods for norm recognition tend to focus only on surface-level features of dialogues and do not take into account the interactions within a conversation. To address this issue, we propose NORMMARK, a probabilistic generative Markov model to carry the latent features throughout a dialogue. These features are captured by discrete and continuous latent variables conditioned on the conversation history, and improve the model's ability in norm recognition. The model is trainable on weakly annotated data using the variational technique. On a dataset with limited norm annotations, we show that our approach achieves higher F1 score, outperforming current stateof-the-art methods, including GPT3.

show abstract

Section: Related Workmentioning

confidence: 99%

NormMark: A Weakly Supervised Markov Model for Socio-cultural Norm Discovery

Moghimifar¹,

Qu²,

Wu³

et al. 2023

Findings of the Association for Computational Linguistics: ACL 2023

View full text Add to dashboard Cite

show abstract

“…"xNeed" and "xIntent" are chosen for CSKG construction, E vv , since they deal with what is needed or intended for the event to occur, while "xWant" and "xEffect" for scoring the natural language actions, E va , since they deal with what the player would do following the event. We further set n hop = 1 and n gen = 2 from the observation that they are good enough for zero-shot commonsense question answering Moghimifar et al, 2020). During the online training of the agent, we freeze the parameters for COMET.…”

Section: B Experiments Setupmentioning

confidence: 99%

Fire Burns, Sword Cuts: Commonsense Inductive Bias for Exploration in Text-based Games

Ryu¹,

Shareghi²,

Fang³

et al. 2022

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

View full text Add to dashboard Cite

Text-based games (TGs) are exciting testbeds for developing deep reinforcement learning techniques due to their partially observed environments and large action spaces. In these games, the agent learns to explore the environment via natural language interactions with the game simulator. A fundamental challenge in TGs is the efficient exploration of the large action space when the agent has not yet acquired enough knowledge about the environment. We propose COMMEXPL, an exploration technique that injects external commonsense knowledge, via a pretrained language model (LM), into the agent during training when the agent is the most uncertain about its next action. Our method exhibits improvement on the collected game scores during the training in four out of nine games from Jericho. Additionally, the produced trajectory of actions exhibit lower perplexity, when tested with a pretrained LM, indicating better closeness to human language. 1 * " = "PerxonX opens the trap door" ! ! + "xEffect" * # = "PerxonX removes the trap door" ! ! + "xWant" * $ = "PerxonX clean up the room" * $ + "xWant" * % = "PerxonX have a clean room" * $ + "xEffect" * & = "PerxonX gets tired from cleaning". ' ()) / ())

show abstract

“…Tony may feel afraid and want to run away because of what he has done wrong. Atomic models are mainly generated by seq2seq 11 (Sequence to Sequence).…”

Section: Knowledge Base Acquisitionmentioning

confidence: 99%

Research on social common sense knowledge reasoning method based on pre-training

Deng

Zhou

et al. 2022

International Conference on Computer Application and Information Security (ICCAIS 2021)

View full text Add to dashboard Cite

Common sense knowledge is a part of knowledge reasoning. At present, common sense knowledge reasoning is also a hot and difficult point in the field of knowledge reasoning. In order to enable machines to use human language, and according to some content reasoning or inference rules, to get the results of common sense reasoning. This paper mainly explains how to obtain the common knowledge base, improve the reasoning ability of the machine to the social common knowledge, and test the common knowledge reasoning data set by using the epoch-making pre-training model through the deep reinforcement learning method. The experiment shows that by adjusting the parameters of the data, the accuracy can be improved, which is also conducive to the realization of the downstream social common sense reasoning question and answer task.

show abstract

CosMo: Conditional Seq2Seq-based Mixture Model for Zero-Shot Commonsense Question Answering

Cited by 5 publications

References 24 publications

NormMark: A Weakly Supervised Markov Model for Socio-cultural Norm Discovery

NormMark: A Weakly Supervised Markov Model for Socio-cultural Norm Discovery

Fire Burns, Sword Cuts: Commonsense Inductive Bias for Exploration in Text-based Games

Research on social common sense knowledge reasoning method based on pre-training

Contact Info

Product

Resources

About