Acquiring core meanings of words, represented as Jackendoff-style conceptual structures, from correlated streams of linguistic and non-linguistic input

Abstract

This paper describes an operational system which can acquire the core meanings of words without any prior knowledge of either the category or meaning of any words it encounters. The system is given as input, a description of sequences of scenes along with sentences which describe the [EVENTS] taking place as those scenes unfold, and produces as output, a lexicon consisting of the category and meaning of each word in the input, that allows the sentences to describe the [EVENTS]. It is argued, that each of the three main components of the system, the parser, the linker and the inference component, make only linguistically and cognitively plausible assumptions about the innate knowledge needed to support tractable learning. The paper discusses the theory underlying the system, the representations and algorithms used in the implementation, the semantic constraints which support the heuristics necessary to achieve tractable learning, the limitations of the current theory and the implications of this work for language acquisition research. GO(z, [y, z]) GO(z, [y, 4) GO(z, [FROM(y), z]) GO(z, [FROM(y), z]) GO(z, [FROM(AT(y)), z]) GO(z, [FROM(AT(y)), z]) GO(z, [y, TO(z)]) GO(x, [y, TO(z)]) GO(z, [FROM(y), TO(z)]) GO(z, [FROM(y), TO(z)]) GO(z, [FROM(AT(y)), TO(z)]) GO(z, [FROM(AT(y)), TO(z)]) GO(z, [y, TO(AT(z))]) GO(z, [y, TO(AT(z))]) GO(z, [FROM(y), TO(AT(z))]) GO(z, [FROM(y), TO(AT(z))]) GO(z, [FROM(AT(y)), TO(AT(z))]) GO(z, [FROM(AT(y)), TO(AT(z))])

The Coordinated Interplay of Scene, Utterance, and World Knowledge: Evidence From Eye Tracking

“…In language acquisition, when parents talk to their children, language is likely often about entities and events in the immediate environment that children typically explore (e.g., Snow, 1977). In these situations, it serves a specific function, namely making the immediate scene accessible for the child, and identifying objects that are relevant for comprehension and acquisition (see, e.g., Richards & Goldfarb, 1986;Roy & Pentland, 2002;Siskind, 1990, for related modeling research).…”

Successful na�ve representation grounding

“…Several workers in Artificial Neural Networks - (Cottrell et al 1990, Plunkett et al 1992, Regier 1992) -in more traditional Artificial Intelligence - (Pustejovsky 1988, Siskind 1990) -and in hybrid systems using both technologies - (Nenov 1991) -have approached the problem of grounding language in sensory stimuli. They have all been successful in some way, but none of them has yet achieved practical applicability in the same way that we think our approach could be said to be practically applicable: it learns in seconds, operates in seconds, and works with real video input, and a variety of linguistic encodings.…”

An empirical study on thematic knowledge acquisition based on syntactic clues and heuristics

“…Extra information resources are needed for thematic knawledge acquisition. From the cognitive point of view, morphological, syntactic, semantic, contextual (Jacobs88), pragmatic, world knowledge, and observations of the environment (Webster89,Siskind90) .~e all important resources. However, the availability~of the resources often deteriorated the feasibility of learning from a practical standpoint.…”