Towards a unifying theory of generalization

Abstract

How do humans generalize from observed to unobserved data? How does generalization support inference, prediction, and decision making? I propose that a big part of human generalization can be explained by a powerful mechanism of function learning. I put forward and assess Gaussian Process regression as a model of human function learning that can unify several psychological theories of generalization. Across 14 experiments and using extensive computational modeling, I show that this model generates testable predictions about human preferences over different levels of complexity, provides a window into compositional inductive biases, and –combined with an optimistic yet efficient sampling strategy– guides human decision making through complex spaces. Chapters 1 and 2 propose that, from a psychological and mathematical perspective, function learning and generalization are close kin. Chapter 3 derives and tests theoretical predictionsof participants’ preferences over differently complex functions. Chapter 4 develops a compositional theory of generalization and extensively probes this theory using 8 experimental paradigms.During the second half of the thesis, I investigate how function learning guides decision making in complex decision making tasks. In particular, Chapter 5 will look at how people search for rewards in various grid worlds where a spatial correlation of rewards provides a context supporting generalization and decision making. Chapter 6 gauges human behavior in contextual multi-armed bandit problems where a function maps features onto expectedrewards. In both Chapter 5 and Chapter 6, I find that the vast majority of subjects are best predicted by a Gaussian Process function learning model combined with an upper confidence bound sampling strategy. Chapter 7 will formally assess the adaptiveness of human generalization in complex decision making tasks using mismatched Bayesian optimization simulations and finds that the empirically observed phenomenon of undergeneralization might rather be a feature than a bug of human behavior. Finally, I summarize the empirical and theoretical lessons learned and lay out a road-map for future research on generalization in Chapter 8.

Zero-shot compositional reasoning in a reinforcement learning setting

“…Six of these are Bayesian models that vary along three dimensions: first, whether or not they can generalize learned values from one option to the other, second, whether or not they use uncertainty-guided exploration strategy, and third, whether or not they can compose the learned values from the first two sub-task to reason on the final sub-task. The Bayesian models include a Bayesian mean-tracker (BMT) 36 which is a model that does not learn about the underlying functional structure but instead updates its beliefs about rewards for each option independently, as well as a model that learns functions by generalizing across options within a sub-task based on the idea of Gaussian Process regression (GPR) 18,37,38 . For each of these two models, we considered one variant that cannot compose and instead learns separate reward functions for each sub-task, another that does not perform uncertainty-guided exploration, and lastly, one that initializes its predictions in the final sub-task to the composition of the learned means from the first two sub-tasks.…”

“…Empirical studies have demonstrated that people have an inherent predisposition towards compositional patterns 7,[18][19][20][21][22][23] . For example, utilizing the function learning paradigm, which involves the learning, completion, and prediction of functional patterns, Schulz and colleagues 18,19 have demonstrated that humans find it easier to learn about compositional than non-compositional patterns. Furthermore, they showed that humans exhibit superior abilities to complete and predict compositional functions, as well as an enhanced capacity for remembering such functions 20,21 .…”