A mixture of generative models strategy helps humans generalize across tasks

Abstract

Humans extract statistical regularities over time by forming internal representations of the transition probabilities between states. Studies on sequence prediction learning typically focus on how experiencing a particular sequence allows the underlying generative model to be learned. Here we ask what role generative models play when participants experience and learn to predict different sequences from a common generative family. Our novel multi-task prediction paradigm requires participants to complete four sequence learning tasks, each task being a different instantiation of a common generative family. This multi-task environment allows participants to demonstrate two types of learning: (i) within-task learning (i.e., finding the solution to each of the tasks), and (ii) across-task learning (i.e., learning from one task that changes how they approach future tasks). Specifically, to examine across-task learning, we focus on the first four trials of each sequence learning task, as choices during these trials must be driven by participants' background knowledge and not by their in-task learning experience. We show not only that participants arrive with a structured prior, but also that as participants proceed from one task to the next, their prior increasingly resembles the true generative family. Then, to analyse the within-task learning, we model behaviour within each task as arising from a mixture-of-generative-models learning strategy, whereby participants have a pool of candidate explanatory models (mostly within the true generative family) that they continuously update and arbitrate against each other to best explain the stimulus sequence. This framework predicts specific structured errors from participants, as well as a gating mechanism for learning, both of which are observed in the data.

Insights about the common generative rule underlying an information foraging task can be facilitated via collective search

“…Social learning is one of the core human capacities that have been vital in both growth and habitat expansion of human populations 8,9,33 . Previous research has shown that humans flexibly use various social learning strategies in response to the adaptive features of an environment 10,11 . Previous research has also used the exploration-exploitation tradeoff in information search 27,28 as a common platform to clarify computational algorithms underlying social learning.…”

“…1 top). If such generalization is warranted (i.e., the old and new environments are structured or generated according to a common rule [10][11][12][13] , decision makers can solve the exploration-exploitation tradeoff in the new environments more efficiently. Although the question of knowledge generalizability has been discussed for many years 14,15 , it has remained largely unanswered because of the computational difficulty of quantifying its cognitive underpinnings in detail.…”

Statistical Learning in Vision

“…Studies in the second group deal with the problem of how to use a structured representation for generalization purposes. These studies investigate how humans generalize through property induction (Kemp & Tenenbaum 2009), how they use learned reward functions for generalization during search tasks in spatially or conceptually correlated and graph-structured reward environments (Castañón et al 2021;Wu et al 2018Wu et al , 2020, and how they can learn how to generalize (Austerweil et al 2019).…”

Thalamus: a brain-inspired algorithm for biologically-plausible continual learning and disentangled representations

“…Animals thrive in a constantly changing environmental demands at many time scales. Biological brains seem capable of using these changes advantageously and leverage the temporal structure to learn causal and well-factorized representations (Collins and Koechlin, 2012;Yu et al, 2021;Herce Castañón et al, 2021). In contrast, traditional neural networks suffer in such settings with sequential experience and display prominent interference between old and new learning limiting most training paradigms to using shuffled data (McCloskey and Cohen, 1989).…”