Goal-Directed Reasoning and Cooperation in Robots in Shared Workspaces: an Internal Simulation Based Neural Framework

Abstract

From social dining in households to product assembly in manufacturing lines, goal-directed reasoning and cooperation with other agents in shared workspaces is a ubiquitous aspect of our day-to-day activities. Critical for such behaviours is the ability to spontaneously anticipate what is doable by oneself as well as the interacting partner based on the evolving environmental context and thereby exploit such information to engage in goal-oriented action sequences. In the setting of an industrial task where two robots are jointly assembling objects in a shared workspace, we describe a bioinspired neural architecture for goal-directed action planning based on coupled interactions between multiple internal models, primarily of the robot’s body and its peripersonal space. The internal models (of each robot’s body and peripersonal space) are learnt jointly through a process of sensorimotor exploration and then employed in a range of anticipations related to the feasibility and consequence of potential actions of two industrial robots in the context of a joint goal. The ensuing behaviours are demonstrated in a real-world industrial scenario where two robots are assembling industrial fuse-boxes from multiple constituent objects (fuses, fuse-stands) scattered randomly in their workspace. In a spatially unstructured and temporally evolving assembly scenario, the robots employ reward-based dynamics to plan and anticipate which objects to act on at what time instances so as to successfully complete as many assemblies as possible. The existing spatial setting fundamentally necessitates planning collision-free trajectories and avoiding potential collisions between the robots. Furthermore, an interesting scenario where the assembly goal is not realizable by either of the robots individually but only realizable if they meaningfully cooperate is used to demonstrate the interplay between perception, simulation of multiple internal models and the resulting complementary goal-directed actions of both robots. Finally, the proposed neural framework is benchmarked against a typically engineered solution to evaluate its performance in the assembly task. The framework provides a computational outlook to the emerging results from neurosciences related to the learning and use of body schema and peripersonal space for embodied simulation of action and prediction. While experiments reported here engage the architecture in a complex planning task specifically, the internal model based framework is domain-agnostic facilitating portability to several other tasks and platforms.Electronic supplementary materialThe online version of this article (10.1007/s12559-018-9553-1) contains supplementary material, which is available to authorized users.

Information Theoretic Model to Simulate Agent-Signage Interaction for Wayfinding

“…In wayfinding research, visual cues in the built environment are a human's primary source of distal information [60]. Indeed, several researchers have developed vision-based techniques for enabling mobile robots and virtual agents to detect and react to information in their environment [61]. In order to realistically model the interaction between agents and their environment, a human-like visual perception model should focus on the first-person perception of signage while considering dynamic occlusions [62].…”

“…The most important features of a biological, cognitive system are the abilities to perceive, act, and learn. Many researchers have built computational models of intelligent cognitive systems [11][12][13] that can cooperate with other agents in a shared workspace towards a common goal [14], perform decision-making during path-planning and avoid obstacles with flying drones [15], and perceive information from an unfamiliar dynamic environment [12]. These abilities can also be characterized in terms of Shannon information theory [16,17].…”

Cognitive System Framework for Brain-Training Exercise Based on Human-Robot Interaction

“…To this end, a Human-Aware Task Planner has been used to define the sequence of actions to perform and to decide whether and when the robot should intervene. Bhat et al [7] present a neural architecture for goal-direct reasoning and cooperation between multiple robots in an industrial task, where two robots work together for assembling objects in a shared workspace.…”

“…The main reason behind a different behaviour is that now the robot, in contrast to the previous case, made the correct moves in the first steps (step 1, 2). It can be noted that even though it considered very little the previous action outcome, in that specific case, because the first level of engagement resulted in a successful strategy for the first two moves of the user, the robot adopted it as a preferred action even in the next steps (4,7,8). The main reason is straightforward if we look at Equation 3.…”