Bayesian analogy with relational transformations.

Abstract

How can humans acquire relational representations that enable analogical inference and other forms of high-level reasoning? Using comparative relations as a model domain, we explore the possibility that bottom-up learning mechanisms applied to objects coded as feature vectors can yield representations of relations sufficient to solve analogy problems. We introduce Bayesian analogy with relational transformations (BART) and apply the model to the task of learning first-order comparative relations (e.g., larger, smaller, fiercer, meeker) from a set of animal pairs. Inputs are coded by vectors of continuousvalued features, based either on human magnitude ratings, normed feature ratings (De Deyne et al., 2008), or outputs of the topics model (Griffiths, Steyvers, & Tenenbaum, 2007). Bootstrapping from empirical priors, the model is able to induce first-order relations represented as probabilistic weight distributions, even when given positive examples only. These learned representations allow classification of novel instantiations of the relations and yield a symbolic distance effect of the sort obtained with both humans and other primates. BART then transforms its learned weight distributions by importance-guided mapping, thereby placing distinct dimensions into correspondence. These transformed representations allow BART to reliably solve 4-term analogies (e.g., larger:smaller::fiercer:meeker), a type of reasoning that is arguably specific to humans. Our results provide a proof-of-concept that structured analogies can be solved with representations induced from unstructured feature vectors by mechanisms that operate in a largely bottom-up fashion. We discuss potential implications for algorithmic and neural models of relational thinking, as well as for the evolution of abstract thought. Keywords: analogy, relation learning, generalization, Bayesian modelsSupplemental materials: http://dx.doi.org/10.1037/a0028719.supp One of the hallmarks of human reasoning is the ability to form representations of relations between entities and then to reason about the higher order relations between these relations. Whereas concepts such as larger and smaller, for example, are first-order relations, potentially derivable by comparing features of individual objects, a relation such as opposite is a higher order relation between relations (Gentner, 1983). The capacity to represent and reason with higher order relations has been considered central to human analogical thinking

The discovery and comparison of symbolic magnitudes

“…The simulation results reported by Lu et al (2012) suggest that concepts related to symbolic magnitudes can be discovered by inductive learning, rather than simply assumed to be directly available in long-term memory. Moreover, the Bayesian approach in general (and the BART model in particular) implies that magnitudes will be represented not as deterministic values, but rather as probability distributions.…”

“…Fig. 1 includes feature vectors for 30 example animals based on the 50 features most highly associated with a larger set of animal names (Lu et al, 2012). Although these ''Leuven vectors'' presumably only approximate people's knowledge about animal concepts, they have the great virtue of being derived from independent sources of data, rather than being hand-coded.…”

“…To provide such distributed representations, Lu et al (2012) developed Bayesian Analogy with Relational Transformations (BART), a model of how one-place scalar adjectives (e.g., large, smart) and two-place comparative relations (e.g., larger, smarter) can be learned from non-relational feature vectors. Using various inputs, including Leuven vectors and vectors derived using the topic model (Griffiths et al, 2007), the model was applied to the acquisition of concepts related to four continuous dimensions: size, ferocity, speed and intelligence.…”

“…The model we propose, termed BARTlet (i.e., the diminutive form of BART), builds on the learning capability of BART (Lu et al, 2012) but makes simpler representational assumptions. A key idea incorporated in both models is that learning can be bootstrapped by incorporating empirical priors-a ''favorable'' initial knowledge state derived from some related but simpler learning task.…”

“…Rather, magnitudes may be formed in working memory based on computations over more basic features stored in long-term memory. We present a model of how magnitudes can be acquired and compared based on BARTlet, a representationally simpler version of Bayesian Analogy with Relational Transformations (BART; Lu, Chen, & Holyoak, 2012). BARTlet operates on distributions of magnitude variables created by applying dimension-specific weights (learned with the aid of empirical priors derived from pre-categorical comparisons) to more primitive features of objects.…”