Integrated Cognitive Architecture for Robot Learning of Action and Language

Miyazawa, Kazuki; Horii, Takato; Aoki, Tatsuya; Nagai, Takayuki

doi:10.3389/frobt.2019.00131

Cited by 19 publications

(13 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As shown by our analysis, all autonomous systems used in user studies were targeting a specific application field. Generic reasoning relates to developmental robotics that can include various theories and methods such as deep learning, sensorimotor information processing, metacognition, memory, and decision making (Miyazawa et al, 2019). Several studies proposed a cognitive architecture for NAO's system design.…”

Section: From Close-loop Systems Towards Real Autonomymentioning

confidence: 99%

10 Years of Human-NAO Interaction Research: A Scoping Review

et al. 2021

View full text Add to dashboard Cite

The evolving field of human-robot interaction (HRI) necessitates that we better understand how social robots operate and interact with humans. This scoping review provides an overview of about 300 research works focusing on the use of the NAO robot from 2010 to 2020. This study presents one of the most extensive and inclusive pieces of evidence on the deployment of the humanoid NAO robot and its global reach. Unlike most reviews, we provide both qualitative and quantitative results regarding how NAO is being used and what has been achieved so far. We analyzed a wide range of theoretical, empirical, and technical contributions that provide multidimensional insights, such as general trends in terms of application, the robot capabilities, its input and output modalities of communication, and the human-robot interaction experiments that featured NAO (e.g. number and roles of participants, design, and the length of interaction). Lastly, we derive from the review some research gaps in current state-of-the-art and provide suggestions for the design of the next generation of social robots.

show abstract

Section: From Close-loop Systems Towards Real Autonomymentioning

confidence: 99%

10 Years of Human-NAO Interaction Research: A Scoping Review

et al. 2021

View full text Add to dashboard Cite

show abstract

“…By connecting perception and action, it is possible to recall the unobserved perception from the action through a simulation p (w v |a). Furthermore, we extend our MLDA to a time-series model, and action-and context-dependent concepts can be learned (Miyazawa et al, 2019) to a certain extent.…”

Section: Definition Of Conceptsmentioning

confidence: 99%

“…Although the MLDA-based approach cannot currently describe all cognitive phenomena, we consider MLDA to be a promising model. In the future, by integrating our previous studies (Fadlil et al, 2013;Miyazawa et al, 2019) and the proposals in this paper, we would like to develop a cognitive model that allows robots to learn by interacting with the environment.…”

Section: Definition Of Conceptsmentioning

confidence: 99%

“…Moreover, we consider that the integration of the MLDA and representation learning is also significant in terms of leveraging past research. Although pre-designed or pre-trained feature extractors have been used, various studies (Attamimi et al, 2014;Nakamura and Nagai, 2017;Hagiwara et al, 2019;Miyazawa et al, 2019) in addition to the above have revealed the effectiveness of the MLDA for multimodal concept formation. That is, the integration of the MLDA and representation learning makes it possible to develop these studies further.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Robot Concept Acquisition Based on Interaction Between Probabilistic and Deep Generative Models

Kuniyasu¹,

Nagaoka²,

Taniguchi³

et al. 2021

Front. Comput. Sci.

Self Cite

View full text Add to dashboard Cite

We propose a method for multimodal concept formation. In this method, unsupervised multimodal clustering and cross-modal inference, as well as unsupervised representation learning, can be performed by integrating the multimodal latent Dirichlet allocation (MLDA)-based concept formation and variational autoencoder (VAE)-based feature extraction. Multimodal clustering, representation learning, and cross-modal inference are critical for robots to form multimodal concepts from sensory data. Various models have been proposed for concept formation. However, in previous studies, features were extracted using manually designed or pre-trained feature extractors and representation learning was not performed simultaneously. Moreover, the generative probabilities of the features extracted from the sensory data could be predicted, but the sensory data could not be predicted in the cross-modal inference. Therefore, a method that can perform clustering, feature learning, and cross-modal inference among multimodal sensory data is required for concept formation. To realize such a method, we extend the VAE to the multinomial VAE (MNVAE), the latent variables of which follow a multinomial distribution, and construct a model that integrates the MNVAE and MLDA. In the experiments, the multimodal information of the images and words acquired by a robot was classified using the integrated model. The results demonstrated that the integrated model can classify the multimodal information as accurately as the previous model despite the feature extractor learning in an unsupervised manner, suitable image features for clustering can be learned, and cross-modal inference from the words to images is possible.

show abstract

“…Here, we do not account this kind of abstraction. (b) A complex unified framework, which integrates multiple modules, for concept and language learning, knowledge acquisition and decision-making has been proposed by Miyazawa et al ( 2019 ). However, their model suffers from two main limitations.…”

Section: Related Workmentioning

confidence: 99%

Learning Actions From Natural Language Instructions Using an ON-World Embodied Cognitive Architecture

2021

View full text Add to dashboard Cite

Endowing robots with the ability to view the world the way humans do, to understand natural language and to learn novel semantic meanings when they are deployed in the physical world, is a compelling problem. Another significant aspect is linking language to action, in particular, utterances involving abstract words, in artificial agents. In this work, we propose a novel methodology, using a brain-inspired architecture, to model an appropriate mapping of language with the percept and internal motor representation in humanoid robots. This research presents the first robotic instantiation of a complex architecture based on the Baddeley's Working Memory (WM) model. Our proposed method grants a scalable knowledge representation of verbal and non-verbal signals in the cognitive architecture, which supports incremental open-ended learning. Human spoken utterances about the workspace and the task are combined with the internal knowledge map of the robot to achieve task accomplishment goals. We train the robot to understand instructions involving higher-order (abstract) linguistic concepts of developmental complexity, which cannot be directly hooked in the physical world and are not pre-defined in the robot's static self-representation. Our proposed interactive learning method grants flexible run-time acquisition of novel linguistic forms and real-world information, without training the cognitive model anew. Hence, the robot can adapt to new workspaces that include novel objects and task outcomes. We assess the potential of the proposed methodology in verification experiments with a humanoid robot. The obtained results suggest robust capabilities of the model to link language bi-directionally with the physical environment and solve a variety of manipulation tasks, starting with limited knowledge and gradually learning from the run-time interaction with the tutor, past the pre-trained stage.

show abstract

Integrated Cognitive Architecture for Robot Learning of Action and Language

Cited by 19 publications

References 39 publications

10 Years of Human-NAO Interaction Research: A Scoping Review

10 Years of Human-NAO Interaction Research: A Scoping Review

Robot Concept Acquisition Based on Interaction Between Probabilistic and Deep Generative Models

Learning Actions From Natural Language Instructions Using an ON-World Embodied Cognitive Architecture

Contact Info

Product

Resources

About