Scalable cognitive modelling: Putting Simon’s (1969) ant back on the beach.

Abstract

A classic goal in cognitive modelling is the integration of process and representation to form complete theories of human cognition (Estes, 1955). This goal is best encapsulated by the seminal work of Simon (1969) who proposed the parable of the ant to describe the importance of understanding the environment that a person is embedded within when constructing theories of cognition. However, typical assumptions in accounting for the role of representation in computational cognitive models do not accurately represent the contents of memory (Johns & Jones, 2010). Recent developments in machine learning and big data approaches to cognition, referred to as scaled cognitive modelling here, offer a potential solution to the integration of process and representation. This article will review standard practices and assumptions that take place in cognitive modelling, how new big data and machine learning approaches modify these practices, and the directions that future research should take. The goal of the article is to ground big data and machine learning approaches that are emerging in the cognitive sciences within classic cognitive theoretical principles to provide a constructive pathway towards the integration of cognitive theory with advanced computational methodology. Public Significance StatementComputational modelling has played a central role in the development of theory in cognitive psychology. Recently, machine learning and big data approaches to understanding cognition have become increasingly popular. This article reviews standard approaches in computational cognitive modelling and specifies how new advanced computational approaches can be used to generate new research pathways in the cognitive sciences.

A comparison of word humor ratings across speakers of North American, British, and Singapore English

“…Recent work by Johns and Jamieson (2018) and Johns (2024) also highlighted how individual variability in language experience affects the variability of word meanings derived from distributional semantic models specifically trained on the language produced by each individual (i.e., high-profile Reddit commenters or book authors). Although such approaches will require large amounts of data in order to ensure stable model estimates (which may be less available for speakers of minority dialects or languages), this research provides an important demonstration that individual variability in one's experienced "language statistics" have a measurable impact on meaning representations, and emphasize the need for cognitive modelling to consider the influence of the individual's experiences and environment on their linguistic performance and representation (Johns et al, 2023).…”

What Can Language Models Tell Us About Human Cognition?

“…Indeed, so uncertain is the current understanding of the latest generation of transformer models that recent work has begun to use cognitive tests to try to determine what is going on inside them (e.g., Binz & Schulz, 2023). However, others have criticized such approaches because they move the goalposts from trying to understand how humans operate to trying to understand why a model behaves like a human ( Johns et al, 2023). More generally, any post hoc explanations of black-box models are likely to be inadequate at best and misleading at worst (Rudin, 2019).…”

“…With enormous quantities of text available on the Internet, it is very easy to allow language models to learn distributional relationships across billions or trillions of words. However, if a model can approximate human behavior using only a corpus that is many times larger-even orders of magnitude larger-than that accumulated in a human's lifetime of language experience, then it is not a plausible model of how linguistic distributional knowledge works in the human mind ( Johns et al, 2023;Warstadt et al, 2023;Wingfield & Connell, 2022). Although people gain a lot of their language experience through spoken language, both in terms of conversation and media consumption (e.g., watching television and movies), the fastest way to accumulate language experience is through reading written texts.…”

“…If a language model aims to account for all of the semantic processing in a task, then it is not a plausible account of how humans perform that task. However, if a language model is instead assumed to implement an essential component of word meaning, which is ultimately grounded in a complementary component of sensorimotor and other experiences, then it offers a cognitively plausible account (Davis & Yee, 2021; Johns et al, 2023; Wingfield & Connell, 2022).…”