Learning Words by Imitating

Abstract

This chapter proposes a single imitation-learning algorithm capable of simultaneously learning linguistic as well as nonlinguistic tasks, without demonstrations being labeled. A human demonstrator responds to an environment that includes the behavior of another human, called the interactant, and the algorithm must learn to imitate this response without being told what the demonstrator was responding to (for example, the position of an object or a speech utterance of the interactant). Since there is no separate symbolic language system, the symbol grounding problem can be avoided/dissolved. The types of linguistic behavior explored are action responses, which includes verb learning but where actions are generalized to include such things as communicative behaviors or internal cognitive operations. Action responses to object positions are learnt in the same way as action responses to speech utterances of an interactant. Three experiments are used to validate the proposed algorithm.

From Language to Motor Gavagai: Unified Imitation Learning of Multiple Linguistic and Nonlinguistic Sensorimotor Skills

“…The computational model presented in this article is an evolution of a previous architecture presented in [2], [3], which considered a model with three agents: a teacher, an interactant and a learner. Here, the learning model has been simplified and made more generic, since no interactant is needed (but yet could be included without significant change for the learner).…”

“…Here, the learning model has been simplified and made more generic, since no interactant is needed (but yet could be included without significant change for the learner). Furthermore, the architecture in [3] was not analyzed in terms of its ability to resolve concurrently multiple kinds of ambiguities as we do here, and its instanciation in [2] assumed a priori specific properties of a linguistic channel, which is not done here.…”

“…Computational approaches to grounded language acquisition have considered the problem of how an embodied and situated learner can infer the meaning of utterances (forms) while observing form-meaning pairs [28], [29], [30], [31], [32], [33], [25], [26], [27], [34], [35], [2]. In many of the models in this area, form-meaning pairs are observed by the learner through interactions with a language teacher (see [33], [35], [3] for detailed reviews). These interactions can often be cast as language games [33], and provide the teacher with learning data equivalent to the process represented in figure 3: In a given context, typically defined as the configurations of the scene around the teacher and learner, the teacher produces a linguistic signal (a symbolic label, a…”