Parallel Distributed Processing (PDP) models have had a profound impact on the study of cognition. One domain in which they have been particularly influential is quasiregular learning, in which mastery requires both learning regularities that capture the majority of the structure in the input plus learning exceptions that violate the regularities. How PDP models learn quasiregularity is still not well understood. Small-and large-scale analyses of a feedforward, three-layer network were carried out to address two fundamental issues about network functioning: how the model can learn both regularities and exceptions without sacrificing generalizability; and the nature of the hidden representation that makes this learning possible. Results show that capacity-limited learning pressures the network to form componential representations, which ensures good generalizability. Small and highly local perturbations of this representational system allow exceptions to be learned while minimally disrupting generalizability. Theoretical and methodological implications of the findings are discussed.
Keywords
PDP model; quasiregularity; network analysis; hidden representationThe Parallel Distributed Processing (PDP) approach to studying cognition has had a significant impact in cognitive science, ranging from the fields of perception to reasoning (Thomas & McClelland, 2008). The premise of the approach is that many neuron-like units interacting through inhibition and excitation can provide insight into not only the end-state of learning, but also crucially the learning process itself. That is, PDP models can be used as tools to understand how mastery of an ability or skill is achieved.A fundamental problem in learning to which PDP models have been applied is learning quasiregularity; that is, learning certain regularities, but also learning violations of those regularities. Quasiregularity is ubiquitous in how humans structure the environment. It is required to form categories of objects, learn concepts, read words, and solve problems. To highlight one example, quasiregularity is especially prevalent in language, and one focus of research in language processing is to understand how humans learn regular items, which can be grouped together and thus described by rules (e.g., verbs that become past tense by Correspondence adding ed), and at the same time learn exceptions, which violate such rules and require a context-sensitive response (e.g., the past tense of find being found, not finded). Rumelhart and McClelland (1987) showed that a PDP model can learn both the regularities and the exceptions.Despite the ability of PDP models to learn quasiregularity, how they do so is poorly understood. In particular, little is known about the representations the model forms that enable it to distinguish regulars from exceptions. More specifically, what is the nature of the transformation between input and output? The complexity of PDP models has made it difficult to answer such questions, yet answers are vital. A model is more useful ...