An Integrated Model of Semantics and Control

Abstract

Understanding the mechanisms enabling the learning and flexible use of knowledge in context-appropriate ways has been a major focus of research in the study of both semantic cognition and cognitive control. We present a unified model of semantics and control that addresses these questions from both perspectives. The model provides a coherent view of how semantic knowledge, and the ability to flexibly access and deploy that knowledge to meet current task demands, arises from end-to-end learning of the statistics of the environment. We show that the model addresses unresolved issues from both literatures, including how control operates over features that covary with one another and how control representations themselves are structured and emerge through learning, through a series of behavioral experiments and simulations. We conclude by discussing the implications of our approach to other fundamental questions in cognitive science, machine learning, and artificial intelligence.

Toward the Emergence of Intelligent Control: Episodic Generalization and Optimization

“…This helped the semantic representations to selectively emphasize the task-relevant feature while minimizing the taskirrelevant feature (upper panels of Figure 12), ultimately improving performance on the task. This finding is consistent with prior work studying the influence of context representations used for control (e.g., Cohen et al, 1990) on the learning of semantic representations (Giallanza et al, 2023), extending those results by providing an explanation both for how context representations emerge through learning and how those can in turn shape the learning of semantic representations.…”

“…In this respect, the EGO framework suggests how a recurrent neural network that represents context, coupled with the storage and similaritybased retrieval of such representations in episodic memory, can explain how control representations emerge through interactions with the environment and learning. This connects the use of context for episodic memory retrieval (e.g., Howard & Kahana, 2002;Polyn & Kahana, 2008) with the use of context for biasing processing in semantic memory (e.g., Giallanza et al, 2023), as in other models of cognitive control. We explore these interactions further in the next study.…”

“…To simulate the task, we augmented the model used in Study 2 with a semantic memory pathway that adds a context-dependent layer directly connecting the stimulus and context to the output. This follows the architecture used in previous neural network models of control (e.g., Cohen et al, 1990) and semantic memory (e.g., Rogers & McClelland, 2004), as well as recent work exploring the relationship between semantics and control (Giallanza et al, 2023). Thus, the model had access to both an episodic pathway (as in Studies 1 and 2) and a semantic pathway that it could use jointly to predict the category for a given stimulus.…”

“…Despite the centrality of control representations to theories of control, little work has focused on the structure of these representations or how they are learned. Recent work has begun to address this question, suggesting that the statistical learning mechanisms responsible for the formation of representations in semantic memory may be critical not only for learning about the structure of the world, but also how to behave within it, shaping the representations used for the control of behavior much as it shapes the representations used to understand the world itself (Giallanza et al, 2023). To date, however, little work has considered whether and how the capabilities afforded by episodic memory may contribute to the formation and use of such representations; that is, how episodic memory may interact with cognitive control.…”

“…First, they can directly facilitate responding to task-relevant stimulus features (e.g., naming the color of a Stroop stimulus; Stroop, 1935) when such responses face competition from otherwise stronger ones (e.g., reading the word in a Stroop stimulus), by selectively biasing the activation of those features (Cohen, 2017;Cohen et al, 1990;Desimone & Duncan, 1995). Second, they can help shape the structure of semantic representations as these are being learned, emphasizing task-relevant distinctions between features to make these easier to selectively activate in the future (Giallanza et al, 2023;Rogers & McClelland, 2004. While previous work has addressed the use of context representations for control in these ways, little work has addressed how such context representations are themselves learned.…”