A mathematical theory of relational generalization in transitive inference

Abstract

Humans and animals routinely infer relations between different items or events and generalize these relations to novel combinations of items ("compositional generalization"). This allows them to respond appropriately to radically novel circumstances and is fundamental to advanced cognition. However, how learning systems (including the brain) can implement the necessary inductive biases has been unclear. Here we investigated transitive inference (TI), a classic relational task paradigm in which subjects must learn a relation (A>B and B>C) and generalize it to new combinations of items (A>C). Through mathematical analysis, we found that a broad range of biologically relevant learning models (e.g. gradient flow or ridge regression) perform TI successfully and recapitulate signature behavioral patterns long observed in living subjects. First, we found that models with item-wise additive representations automatically encode transitive relations. Second, for more general representations, a single scalar "conjunctivity factor" determines model behavior on TI and, further, the principle of norm minimization (a standard statistical inductive bias) enables models with fixed, partly conjunctive representations to generalize transitively. Finally, neural networks in the "rich regime," which enables representation learning and often leads to better generalization, deviate in task behavior from living subjects and can make generalization errors. Our findings show systematically how minimal statistical learning principles can explain the rich behaviors empirically observed in TI in living subjects, uncover the mechanistic basis of transitive generalization in standard learning models, and lay out a formally tractable approach to understanding the neural basis of relational generalization.