In coming to understand the world-in learning concepts, acquiring language, and grasping causal relations-our minds make inferences that appear to go far beyond the data available. How do we do it? This review describes recent approaches to reverse-engineering human learning and cognitive development and, in parallel, engineering more humanlike machine learning systems. Computational models that perform probabilistic inference over hierarchies of flexibly structured representations can address some of the deepest questions about the nature and origins of human thought: How does abstract knowledge guide learning and reasoning from sparse data? What forms does our knowledge take, across different domains and tasks? And how is that abstract knowledge itself acquired?

No abstract

Algorithms for finding structure in data have become increasingly important both as tools for scientific data analysis and as models of human learning, yet they suffer from a critical limitation. Scientists discover qualitatively new forms of structure in observed data: For instance, Linnaeus recognized the hierarchical organization of biological species, and Mendeleev recognized the periodic structure of the chemical elements. Analogous insights play a pivotal role in cognitive development: Children discover that object category labels can be organized into hierarchies, friendship networks are organized into cliques, and comparative relations (e.g., ''bigger than'' or ''better than'') respect a transitive order. Standard algorithms, however, can only learn structures of a single form that must be specified in advance: For instance, algorithms for hierarchical clustering create tree structures, whereas algorithms for dimensionality-reduction create low-dimensional spaces. Here, we present a computational model that learns structures of many different forms and that discovers which form is best for a given dataset. The model makes probabilistic inferences over a space of graph grammars representing trees, linear orders, multidimensional spaces, rings, dominance hierarchies, cliques, and other forms and successfully discovers the underlying structure of a variety of physical, biological, and social domains. Our approach brings structure learning methods closer to human abilities and may lead to a deeper computational understanding of cognitive development.cognitive development ͉ structure discovery ͉ unsupervised learning

No abstract

Inductive learning is impossible without overhypotheses, or constraints on the hypotheses considered by the learner. Some of these overhypotheses must be innate, but we suggest that hierarchical Bayesian models help explain how the rest can be acquired. To illustrate this claim, we develop models that acquire two kinds of overhypotheses -overhypotheses about feature variability (e.g. the shape bias in word learning) and overhypotheses about the grouping of categories into ontological kinds like objects and substances. Learning overhypotheses 3Learning overhypotheses with hierarchical Bayesian models Compared to our best formal models, children are remarkable for learning so much from so little. A single labelled example is enough for children to learn the meanings of some words (Carey & Bartlett, 1978), and children develop grammatical constructions that are rarely found in the sentences that they hear (Chomsky, 1980). These inductive leaps appear even more impressive when we consider the many interpretations of the data that are logically possible but apparently never entertained by children (Goodman, 1955;Quine, 1960).Learning is impossible without constraints of some sort, but the apparent ease of children's learning may rely on relatively strong inductive constraints. Researchers have suggested, for example, that the M-constraint (Keil, 1979) and the shape bias (Heibeck & Markman, 1987) help explain concept learning, that universal grammar guides the acquisition of linguistic knowledge (Chomsky, 1980), and that constraints on the properties of physical objects (Spelke, 1990) support inferences about visual scenes.Constraints like these may be called theories or schemata, but we will borrow a term of Goodman's and refer to them as overhypotheses. 1 Although overhypotheses play a prominent role in nativist approaches to development (Keil, 1979;Chomsky, 1980;Spelke, 1990), some overhypotheses are probably learned (Goldstone & Johansen, 2003). One such overhypothesis is the shape bias -the expectation that all of the objects in a given category tend to have the same shape, even if they differ along other dimensions, such as color and texture. Smith, Jones, Landau, Gershkoff-Stowe, and Samuelson (2002) provide strong evidence that the shape bias is learned by showing that laboratory training allows children to demonstrate this bias at an age before it normally emerges. Other overhypotheses that appear to be learned include constraints on the rhythmic pattern of a child's native language (Jusczyk, 2003), Learning overhypotheses 4 and constraints on the kinds of feature correlations that are worth tracking when learning about artifacts or other objects (Madole & Cohen, 1995).The acquisition of overhypotheses raises some difficult challenges for formal models.It is difficult at first to understand how something as abstract as an overhypothesis might be learned, and the threat of an infinite regress must also be confronted -what are the inductive constraints that allow inductive constraints to be learned? ...