Abstract-This paper introduces a cognitive architecture for a humanoid robot to engage in a proactive, mixed-initiative exploration and manipulation of its environment, where the initiative can originate from both the human and the robot. The framework, based on a biologically-grounded theory of the brain and mind, integrates a reactive interaction engine, a number of state-of-the art perceptual and motor learning algorithms, as well as planning abilities and an autobiographical memory. The architecture as a whole drives the robot behavior to solve the symbol grounding problem, acquire language capabilities, execute goal-oriented behavior, and express a verbal narrative of its own experience in the world. We validate our approach in humanrobot interaction experiments with the iCub humanoid robot, showing that the proposed cognitive architecture can be applied in real time within a realistic scenario and that it can be used with naive users.

It has been proposed that starting from meaning that the child derives directly from shared experience with others, adult narrative enriches this meaning and its structure, providing causal links between unseen intentional states and actions. This would require a means for representing meaning from experience—a situation model—and a mechanism that allows information to be extracted from sentences and mapped onto the situation model that has been derived from experience, thus enriching that representation. We present a hypothesis and theory concerning how the language processing infrastructure for grammatical constructions can naturally be extended to narrative constructions to provide a mechanism for using language to enrich meaning derived from physical experience. Toward this aim, the grammatical construction models are augmented with additional structures for representing relations between events across sentences. Simulation results demonstrate proof of concept for how the narrative construction model supports multiple successive levels of meaning creation which allows the system to learn about the intentionality of mental states, and argument substitution which allows extensions to metaphorical language and analogical problem solving. Cross-linguistic validity of the system is demonstrated in Japanese. The narrative construction model is then integrated into the cognitive system of a humanoid robot that provides the memory systems and world-interaction required for representing meaning in a situation model. In this context proof of concept is demonstrated for how the system enriches meaning in the situation model that has been directly derived from experience. In terms of links to empirical data, the model predicts strong usage based effects: that is, that the narrative constructions used by children will be highly correlated with those that they experience. It also relies on the notion of narrative or discourse function words. Both of these are validated in the experimental literature.

How do people learn to talk about the causal and temporal relations between events, and the motivation behind why people do what they do? The narrative practice hypothesis of Hutto and Gallagher holds that children are exposed to narratives that provide training for understanding and expressing reasons for why people behave as they do. In this context, we have recently developed a model of narrative processing where a structured model of the developing situation (the situation model) is built up from experienced events, and enriched by sentences in a narrative that describe event meanings. The main interest is to develop a proof of concept for how narrative can be used to structure, organize and describe experience. Narrative sentences describe events, and they also define temporal and causal relations between events. These relations are specified by a class of narrative function words, including “because, before, after, first, finally.” The current research develops a proof of concept that by observing how people describe social events, a developmental robotic system can begin to acquire early knowledge of how to explain the reasons for events. We collect data from naïve subjects who use narrative function words to describe simple scenes of human-robot interaction, and then employ algorithms for extracting the statistical structure of how narrative function words link events in the situation model. By using these statistical regularities, the robot can thus learn from human experience about how to properly employ in question-answering dialogues with the human, and in generating canonical narratives for new experiences. The behavior of the system is demonstrated over several behavioral interactions, and associated narrative interaction sessions, while a more formal extended evaluation and user study will be the subject of future research. Clearly this is far removed from the power of the full blown narrative practice capability, but it provides a first step in the development of an experimental infrastructure for the study of socially situated narrative practice in human-robot interaction.

With the Robonaut-2 humanoid robot now permanently flying on the ISS, the potential role for robots participating in cooperative activity in space is becoming a reality. Recent research has demonstrated that cooperation in the joint achievement of shared goals is a promising framework for human interaction with robots, with application in space. Perhaps more importantly, with the turn-over of crew members, robots could play an important role in maintaining and transferring expertise between outgoing and incoming crews. In this context, the current research builds on our experience in systems for cooperative human-robot interaction, introducing novel interface and interaction modalities that exploit the long-term experience of the robot. We implement a system where the human agent can teach the Nao humanoid new actions by physical demonstration, visual imitation, and spoken command. These actions can then be composed into joint action plans that coordinate the cooperation between agent and human.We also implement algorithms for an Autobiographical Memory (ABM) that provides access to of all of the robots interaction experience.These functions are assembled in a novel interaction paradigm for the capture, maintenance and transfer of knowledge in a five-tiered structure. The five tiers allow the robot to 1) learn simple behaviors, 2) learn shared plans composed from the learned behaviors, 3) execute the learned shared plans efficiently, 4) teach shared plans to new humans, and 5) answer questions from the human to better understand the origin of the shared plan. Our results demonstrate the feasibility of this system and indicate that such humanoid robot systems will provide a potential mechanism for the accumulation and transfer of knowledge, between humans who are not co-present. Applications to space flight operations as a target scenario are discussed.Index Terms-human-robot interaction, shared-plan, behavior learning, robotic teaching, space-flight operations.