On Semantic Cognition, Inductive Generalization, and Language Models

Abstract

My doctoral research focuses on understanding semantic knowledge in neural network models trained solely to predict natural language (referred to as language models, or LMs), by drawing on insights from the study of concepts and categories grounded in cognitive science. I propose a framework inspired by 'inductive reasoning,' a phenomenon that sheds light on how humans utilize background knowledge to make inductive leaps and generalize from new pieces of information about concepts and their properties. Drawing from experiments that study inductive reasoning, I propose to analyze semantic inductive generalization in LMs using phenomena observed in human-induction literature, investigate inductive behavior on tasks such as implicit reasoning and emergent feature recognition, and analyze and relate induction dynamics to the learned conceptual representation space.

An experimental study measuring the generalization of fine‐tuned language representation models across commonsense reasoning benchmarks

“…Ultimately, such language representation models would be used in the real world, not only in multiple‐choice settings, but also in so‐called generative settings where the model may be expected to generate answers to questions (without being given options). Even in the multiple‐choice setting, without robust commonsense, the model will likely not be usable for actual decision making unless we can trust that it is capable of generalization (Kejriwal, 2021; Misra, 2022; Wahle et al, 2022). One option to implementing such robustness in practice may be to add a ‘decision‐making layer’ on a pre‐trained language representation model rather than aim to modify the model's architecture from scratch 13 (Hong et al, 2021; Tang & Kejriwal, 2022; Zaib et al, 2020).…”

Can language representation models think in bets?

“…While some work has found that information can be recovered from BERT's token representation [67], the model still has trouble 'understanding' concepts that are relatively natural to humans, such as negation and basic numeracy [25]. Like many other machine learning models, the model can also be overly confident in some of its inputs, and is susceptible to problems of both generalization and adversarial attacks [34,43,[68][69][70]. Furthermore, several experiments have demonstrated that, although BERT effectively encodes information about relations, entity types, relations, semantic roles, as well as proto-roles, it can lose some of its robustness in the face of basic named entity replacements [26].…”