Commonsense Knowledge Reasoning and Generation with Pre-trained Language Models: A Survey

Bhargava, Prajjwal; Ng, Vincent

doi:10.48550/arxiv.2201.12438

Cited by 4 publications

(5 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…4000 sentence pairs Crowd sourcing CosmosQA [63] Question answering 35,600 problems Crowd sourcing CommonsenseQA [130] Question answering 12,247 questions Crowd sourcing CommonsenseQA 2.0 [131] Yes/no questions 14,343 questions Gamification COPA [111] Select a conclusion causally 1000 questions Expert authors connected to a premise CREAK [104] True/False questions 13,000 questions Crowd sourcing CycIC (No paper) Question answering 10,700 questions Synthesized DefeasibleNLI [114] Does new information 250,000 examples Crowd sourcing strenghthen an inference? DiscoSense [11] Understanding of 13,056 examples Adapted discourse markers ETHICS [59] Ethical judgments 130,000+ examples Crowd sourcing FeasibilityQA [57] Question answering 4608 questions Expert construction. GeoMLAMA [150] Question answering 3145 questions Expert construction Multilingual en zh hi fa sw HellaSwag [156] What will 70,000 questions Adapted & happen next?…”

Section: Collections Of Elementary Mathematical Word Problemsmentioning

confidence: 99%

“…When someone wants to test their AI on your benchmark, they train on your published training set, then they upload their trained model onto your machine, and you can run their model on your test set without exposing its contents. This approach was taken, for instance, in the long series of NLP challenges organized by NIST and by the leaderboards for various tasks administered by the Allen AI Institute 11. Since the training set and test set are random samples of the same corpus, they should have essentially identical features.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Benchmarks for Automated Commonsense Reasoning: A Survey

Davis¹

2023

Preprint

View full text Add to dashboard Cite

More than one hundred benchmarks have been developed to test the commonsense knowledge and commonsense reasoning abilities of artificial intelligence (AI) systems. However, these benchmarks are often flawed and many aspects of common sense remain untested. Consequently, we do not currently have any reliable way of measuring to what extent existing AI systems have achieved these abilities.This paper surveys the development and uses of AI commonsense benchmarks. We discuss the nature of common sense; the role of common sense in AI; the goals served by constructing commonsense benchmarks; and desirable features of commonsense benchmarks. We analyze the common flaws in benchmarks, and we argue that it is worthwhile to invest the work needed ensure that benchmark examples are consistently high quality. We survey the various methods of constructing commonsense benchmarks. We enumerate 139 commonsense benchmarks that have been developed: 102 text-based, 18 image-based, 12 video based, and 7 simulated physical environments. We discuss the gaps in the existing benchmarks and aspects of commonsense reasoning that are not addressed in any existing benchmark. We conclude with a number of recommendations for future development of commonsense AI benchmarks.

show abstract

Section: Collections Of Elementary Mathematical Word Problemsmentioning

confidence: 99%

mentioning

confidence: 99%

Benchmarks for Automated Commonsense Reasoning: A Survey

Davis¹

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Another line of work proposes the evaluation of LLMs on downstream tasks exploiting well-crafted datasets, each of which is dedicated on different reasoning senses. To this end, various tests have stressed LLM capabilities on arithmetic [59], symbolic [58], commonsense [60], and other types of reasoning. Overall, the findings occurring from the aforementioned endeavors suggest that indeed, LLMs present emergent reasoning capabilities simulating human thinking patterns, though being incapable of tackling complex reasoning challenges.…”

Section: Reasoning In Knowledge Graphs and Large Language Modelsmentioning

confidence: 99%

The Contribution of Knowledge in Visiolinguistic Learning: A Survey on Tasks and Challenges

Lymperaiou¹,

Stamou²

2023

Preprint

View full text Add to dashboard Cite

Recent advancements in visiolinguistic (VL) learning have allowed the development of multiple models and techniques that offer several impressive implementations, able to currently resolve a variety of tasks that require the collaboration of vision and language. Current datasets used for VL pre-training only contain a limited amount of visual and linguistic knowledge, thus significantly limiting the generalization capabilities of many VL models. External knowledge sources such as knowledge graphs (KGs) and Large Language Models (LLMs) are able to cover such generalization gaps by filling in missing knowledge, resulting in the emergence of hybrid architectures. In the current survey, we analyze tasks that have benefited from such hybrid approaches. Moreover, we categorize existing knowledge sources and types, proceeding to discussion regarding the KG vs LLM dilemma and its potential impact to future hybrid approaches.

show abstract

“…After pretraining a language model (LM), it undoubtedly acquires linguistic knowledge but also factual (facts about entities), relational (relation between concepts/entities), and commonsense knowledge [32,35]. However, they also tend to suffer from some issues such as (1) catastrophic forgetting after fine-tuning on a downstream task despite any type of regularization, (2) the hallucination problem, for example in dialogue systems (pretrained LMs generate factually incorrect statements) [8,34], and (3) the fact that they rely on memorization during pretraining, which makes them struggle with unseen entities [4,21]. To alleviate these issues, a certain number of recent approaches focused on solutions to better implicitly incorporate knowledge in Pretrained Language Models (PLMs).…”

Section: Knowledge Injection Into Language Modelsmentioning

confidence: 99%

Explicit Knowledge Integration for Knowledge-Aware Visual Question Answering about Named Entities

Adjali,

Grimal,

Ferret

et al. 2023

Proceedings of the 2023 ACM International Conference on Multimedia Retrieval

View full text Add to dashboard Cite

Recent years have shown unprecedented growth of interest in Vision-Language related tasks, with the need to address the inherent challenges of integrating linguistic and visual information to solve real-world applications. Such a typical task is Visual Question Answering (VQA), which aims to answer questions about visual content. The limitations of the VQA task in terms of question redundancy and poor linguistic variability encouraged researchers to propose Knowledge-aware Visual Question Answering tasks as a natural extension of VQA. In this paper, we tackle the KVQAE (Knowledge-based Visual Question Answering about named Entities) task, which proposes to answer questions about named entities defined in a knowledge base and grounded in visual content. In particular, besides the textual and visual information, we propose to leverage the structural information extracted from syntactic dependency trees and external knowledge graphs to help answer questions about a large spectrum of entities of various types. Thus, by combining contextual and graph-based representations using Graph Convolutional Networks (GCNs), we are able to learn meaningful embeddings for Information Retrieval tasks. Experiments on the ViQuAE public dataset show how our approach improves the state-of-the-art baselines while demonstrating the interest of injecting external knowledge to enhance multimodal information retrieval. CCS CONCEPTS• Information systems → Question answering; Test collections; Multimedia and multimodal retrieval.

show abstract

Commonsense Knowledge Reasoning and Generation with Pre-trained Language Models: A Survey

Cited by 4 publications

References 0 publications

Benchmarks for Automated Commonsense Reasoning: A Survey

Benchmarks for Automated Commonsense Reasoning: A Survey

The Contribution of Knowledge in Visiolinguistic Learning: A Survey on Tasks and Challenges

Explicit Knowledge Integration for Knowledge-Aware Visual Question Answering about Named Entities

Contact Info

Product

Resources

About