Packing: A geometric analysis of feature selection and category formation

Abstract

This paper presents a geometrical analysis of how local interactions in a large population of categories packed into a feature space create a global structure of feature relevance. The theory is a formal proof that the joint optimization of discrimination and inclusion creates a smooth space of categories such that near categories in the similarity space have similar generalization gradients. Packing theory offers a unified account of several phenomena in human categorization including the differential importance of different features for different kinds of categories, the dissociation between judgments of similarity and judgments of category membership, and children's ability to generalize a category from very few examples.

A Single Word in a Population of Words

“…Packing Theory (as currently formulated, Hidaka & Smith, 2008,2009) my provide new insights into a some aspects of these results: How children use the perceptual features of things, such as having eyes, being angular, being solid to select other features such as similarity in shape or in texture, thereby enabling children to systematically generalize names for different kinds of things in different ways. The applilcability of Packing Theory to this developmental phenomenon begins with the fact that children’s novel noun generalizations for eyed and non-eyed things and solid and nonsolid things appear to directly reflect the feature distributions within the noun categories that children typically learn early.…”

“…Packing Theory (Hidaka & Smith, 2008, 2009) is an answer to the question of why categories that are near each other in some feature space might have similar generalization patterns. The first insight is that this does not have to be the case, but is likely to be the case under some simple geometric constraints.…”

“…This kind of competitive process is common to many psychological theories (Huttenlocher et al, 2007; Ashby & Valentin, 2005, Kohonen, 1995). Packing Theory also proposes that because of this competition, the learner decreases the probability that an instance is in a category in relation to its ambiguity with respect to neighboring categories (see Hidaka & Smith, 2008, 2009, for the formal specification of this joint optimization of inclusion and discrimination). The local competition results in an estimated category distribution that distorts the experienced distribution as shown by the dotted lines: there is a shift in the psychological distribution of instances that optimizes inclusion of experienced instances and the discrimination of instances associated with different categories.…”

“…Moreover, two categories that compete with each other in a local region in a feature space influence the whole structure by chains of category interactions . The mathematical formulation of Packing theory considers the dynamics of category inclusion and discrimination in a general N-category case and specifies the stable optimal state (see Hidaka & Smith, 2008, 2009, for the formal analysis). The key fact is that the result is a space of categories much like that in Figure 1; there is a global gradient of changing alignments of the generalization pattern such that nearer categories are more similar in their alignment but farther categories are less similar in their distribution of instances in the feature space.…”

“…The proposed mechanism is a general one in two senses, First, it is based on general processes relevant to any form of category learning: the discrimination of instances that belong in different categories and the inclusion of experienced instances within a category. Second, a formal analytic proof (Hidaka & Smith, 2008, 2009) shows that –within a space of many known instances and categories -- the joint optimization of discrimination and inclusion is sufficient to create a space of lexical categories that constrains the possible extensions of a new category as it “fits in” to that space. One goal of this paper is to bring the insights of that mathematical analysis to researchers of children’s word learning.…”