PaCo: Preconditions Attributed to Commonsense Knowledge

Abstract

The task of identifying and reasoning with circumstantial preconditions associated with everyday facts is natural to humans. It is unclear whether state-of-the-art language models (LMs) understand the implicit preconditions that enable or invalidate commonsense facts, such as A glass is used for drinking water, Despite their impressive accuracy on existing commonsense tasks. In this paper, we propose a new problem of reasoning with circumstantial preconditions, and present a dataset, called CoreQuisite, which annotates commonsense facts with preconditions expressed in natural language. Based on this resource, we create three canonical evaluation tasks and use them to examine the capability of existing LMs to understand situational preconditions. Our results show that there is a 10-30% gap between machine and human performance on our tasks. We make all resources and software publicly available.

Factors affecting students’ speaking performance in English department of Jawzjan University

“…Considering that the FLAVA has been pretrained on a vast corpus, our result shows the novelty and uniqueness of the PVLI task. This result is consistent with the similar analysis in Qasemi et al (2022a), for comparing MNLI task with PNLI (text-only).…”

“…Preconditions of Commonsense Knowledge reasoning with preconditions of common sense has been studied in the context of affordance in different fields from cognitive sciences (Garbarini and Adenzato, 2004) to robotics (Ahn et al, 2022) but was recently brought up in natural language understanding. In NLP, the focus has been mainly on proposing human-verified learning resources (Qasemi et al, 2022a;Sap et al, 2019;Heindorf et al, 2020;Do and Pavlick, 2021;Jiang et al, 2021a). Among them, Qasemi et al (2022a) and propose variations of the canonical NLI task for preconditioned inference in common sense.…”

“…For example, a glass may be used for drinking water, under an implicit assumption that the water is at normal temperature, but may not be if the glass is shattered. From the cognitive perspective, understanding the affordance of objects, or simply preconditions of actions (Qasemi et al, 2022a), is part of the commonsense knowledge that constitutes what distinguishes humans from a machine to make inference (Lenat, 1998). From an applications perspective, it also has huge implications such as robotics (Ahn et al, 2022), trans-Figure 1: Preconditioned Visual Language Inference (PVLI) and Preconditioned Visual Language Reasoning (PVLR) tasks.…”

“…preconditioned inference), is proposed as a benchmarking task for evaluation of the theory of affordance (Qasemi et al, 2022a). Multiple studies have formulated the preconditioned natural language inference (PNLI) as variations of the Natural Language Inference (NLI) (Williams et al, 2018;Bowman et al, 2015a;Condoravdi et al, 2003) task and contributed learning resources that are gathered through crowdsouring Qasemi et al, 2022a;Do and Pavlick, 2021;Jiang et al, 2021b) or weak supervision data . In PNLI, the models rely on the contextual information (i.e.…”

“…In the second strategy, Caption Querying, we use the existing crowdsourced PNLI instances (e.g. ; Qasemi et al (2022a);; Do and Pavlick (2021); Jiang et al (2021b)) and find an image caption that is semantically identical to them. The third strategy, Image Querying, focuses solely on the PNLI instances and devises queries (such as "you are in a desert") to search directly for corresponding images on the web using image search engines (e.g.…”