From Language to Motor Gavagai: Unified Imitation Learning of Multiple Linguistic and Nonlinguistic Sensorimotor Skills

Cederborg, Thomas; Oudeyer, Pierre-Yves

doi:10.1109/tamd.2013.2279277

Cited by 4 publications

(6 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The model's behavior is adapted to make the sensory consequences of its actions better match sensory model learned from watching the tutor's actions. Cederborg and Oudeyer (2013) introduced a model for learning to acquire multiple skills by observing a tutor's ambiguous demonstrations. The model integrates concepts and techniques from earlier cross-situational learning models, as well as models of motor learning by demonstration that treat meanings as complex sensorimotor policies with coordinate systems that must be inferred.…”

Section: Figurementioning

confidence: 99%

Computational and Robotic Models of Early Language Development

Oudeyer¹,

Kachergis

Schueller³

2019

International Handbook of Language Acquisition

Self Cite

View full text Add to dashboard Cite

We review computational and robotics models of early language learning and development. We first explain why and how these models are used to understand better how children learn language. We argue that they provide concrete theories of language learning as a complex dynamic system, complementing traditional methods in psychology and linguistics. We review different modeling formalisms, grounded in techniques from machine learning and artificial intelligence such as Bayesian and neural network approaches. We then discuss their role in understanding several key mechanisms of language development: cross-situational statistical learning, embodiment, situated social interaction, intrinsically motivated learning, and cultural evolution. We conclude by discussing future challenges for research, including modeling of large-scale empirical data about language acquisition in real-world environments.

show abstract

Section: Figurementioning

confidence: 99%

Computational and Robotic Models of Early Language Development

Oudeyer¹,

Kachergis

Schueller³

2019

International Handbook of Language Acquisition

Self Cite

View full text Add to dashboard Cite

show abstract

“…Thus, learning is restricted to some tasks within a specific setting. In the case where multiple tasks can be learned in one frame, learning is slowed down considerably by statistical inference and could be quicker if richer information about the social interaction could be exploited (cf., Cederborg and Oudeyer ( 2013 )). Consider the example of a common state-of-the-art imitation learning approach we presented above: in Calinon et al ( 2010 ), the authors describe an experiment in which a humanoid robot is taught to feed a doll.…”

Section: Review Of Teaching/learning Frames Used In the Robot Learninmentioning

confidence: 99%

“…Even when pragmatic frames are already known, a respective architecture would entail using low-level learning mechanisms to acquire these target skills that can be adequately parameterized to benefit from the information contained in the interactional structure to bias their statistical inference (e.g., algorithms for learning motor skills should be able to get information about what aspects of the demonstrated behavior are important based on the interactional cues). For learning sensorimotor skills, Gaussian Mixture Models (or similar probabilistic models) could be used as a method to acquire new target motor skills, such as in state-of-the art methods for robot learning by demonstration [both for motor skills (Calinon and Billard, 2007 ) and language skills (Cederborg and Oudeyer, 2013 )]. To acquire the meaning of new words, Bayesian inference techniques such as those presented in Xu and Tenenbaum ( 2007 ) could be used.…”

Section: Perspectives and Challenges For Future Researchmentioning

confidence: 99%

“…The first challenge is to develop adequate representations of the space of frame meanings so that it can be used operationally to bias the inference of a statistical learning algorithm used to learn a target skill or a target word. A possibility could be to use a Bayesian framework, where at the low-level, Gaussian Mixture Models (or similar probabilistic models) can be used as method to acquire new target motor skills or word meaning, such as in state-of-the art methods for robot learning by demonstration (both for motor skills (Calinon and Billard, 2007 ) and language skills (Cederborg and Oudeyer, 2013 )). Such methods could allow to encode the meaning of frames as Bayesian priors over the space of motor skills or new words, and multiplicative operations over these priors are naturally capable of encoding the combination of multiple priors (such as, for example, when the meaning of a frame encodes an information such as “the target concept is a movement of the hand and the demonstration shows the goal”).…”

Section: Perspectives and Challenges For Future Researchmentioning

confidence: 99%

See 1 more Smart Citation

Pragmatic Frames for Teaching and Learning in Human–Robot Interaction: Review and Challenges

et al. 2016

Self Cite

View full text Add to dashboard Cite

One of the big challenges in robotics today is to learn from human users that are inexperienced in interacting with robots but yet are often used to teach skills flexibly to other humans and to children in particular. A potential route toward natural and efficient learning and teaching in Human-Robot Interaction (HRI) is to leverage the social competences of humans and the underlying interactional mechanisms. In this perspective, this article discusses the importance of pragmatic frames as flexible interaction protocols that provide important contextual cues to enable learners to infer new action or language skills and teachers to convey these cues. After defining and discussing the concept of pragmatic frames, grounded in decades of research in developmental psychology, we study a selection of HRI work in the literature which has focused on learning–teaching interaction and analyze the interactional and learning mechanisms that were used in the light of pragmatic frames. This allows us to show that many of the works have already used in practice, but not always explicitly, basic elements of the pragmatic frames machinery. However, we also show that pragmatic frames have so far been used in a very restricted way as compared to how they are used in human–human interaction and argue that this has been an obstacle preventing robust natural multi-task learning and teaching in HRI. In particular, we explain that two central features of human pragmatic frames, mostly absent of existing HRI studies, are that (1) social peers use rich repertoires of frames, potentially combined together, to convey and infer multiple kinds of cues; (2) new frames can be learnt continually, building on existing ones, and guiding the interaction toward higher levels of complexity and expressivity. To conclude, we give an outlook on the future research direction describing the relevant key challenges that need to be solved for leveraging pragmatic frames for robot learning and teaching.

show abstract

“…On the other hand, many models of learning semantic components from one modality also encounter similar ambiguity issues. For exampe, Cederborg and Oudeyer [ 12 ] draw a parallel between Quine’s inderterminacy and ambiguity in imitation learning, that they call the motor gavagai problem . Another example is encountered with concepts that corresponds to categories.…”

Section: Introductionmentioning

confidence: 99%

MCA-NMF: Multimodal Concept Acquisition with Non-Negative Matrix Factorization

et al. 2015

Self Cite

View full text Add to dashboard Cite

In this paper we introduce MCA-NMF, a computational model of the acquisition of multimodal concepts by an agent grounded in its environment. More precisely our model finds patterns in multimodal sensor input that characterize associations across modalities (speech utterances, images and motion). We propose this computational model as an answer to the question of how some class of concepts can be learnt. In addition, the model provides a way of defining such a class of plausibly learnable concepts. We detail why the multimodal nature of perception is essential to reduce the ambiguity of learnt concepts as well as to communicate about them through speech. We then present a set of experiments that demonstrate the learning of such concepts from real non-symbolic data consisting of speech sounds, images, and motions. Finally we consider structure in perceptual signals and demonstrate that a detailed knowledge of this structure, named compositional understanding can emerge from, instead of being a prerequisite of, global understanding. An open-source implementation of the MCA-NMF learner as well as scripts and associated experimental data to reproduce the experiments are publicly available.

show abstract

From Language to Motor Gavagai: Unified Imitation Learning of Multiple Linguistic and Nonlinguistic Sensorimotor Skills

Cited by 4 publications

References 60 publications

Computational and Robotic Models of Early Language Development

Computational and Robotic Models of Early Language Development

Pragmatic Frames for Teaching and Learning in Human–Robot Interaction: Review and Challenges

MCA-NMF: Multimodal Concept Acquisition with Non-Negative Matrix Factorization

Contact Info

Product

Resources

About