MaestROB: A Robotics Framework for Integrated Orchestration of Low-Level Control and High-Level Reasoning

Abstract

This paper describes a framework called Mae-stROB. It is designed to make the robots perform complex tasks with high precision by simple high-level instructions given by natural language or demonstration. To realize this, it handles a hierarchical structure by using the knowledge stored in the forms of ontology and rules for bridging among different levels of instructions. Accordingly, the framework has multiple layers of processing components; perception and actuation control at the low level, symbolic planner and Watson APIs for cognitive capabilities and semantic understanding, and orchestration of these components by a new open source robot middleware called Project Intu at its core. We show how this framework can be used in a complex scenario where multiple actors (human, a communication robot, and an industrial robot) collaborate to perform a common industrial task. Human teaches an assembly task to Pepper (a humanoid robot from SoftBank Robotics) using natural language conversation and demonstration. Our framework helps Pepper perceive the human demonstration and generate a sequence of actions for UR5 (collaborative robot arm from Universal Robots), which ultimately performs the assembly (e.g. insertion) task.

A Survey on Multi-Robot Coordination in Electromagnetic Adversarial Environment: Challenges and Techniques

“…Nao. In [125], a hierarchical robotics framework MaestROS is proposed where different components can be orchestrated to perform complex behaviors. MaestROS can support high-level instruction inputs, such as natural language and demonstrations, and can train the robots to perform a task with these instructions.…”

Enabling Human-Like Task Identification From Natural Conversation

“…This lessens the effort to re-use our system in a new domain. Though Lu et al [25] and Munawar et al [26] proposed approaches to convert the parsed semantic information to a formal language for planners, we extend this by a complete framework that uses human-robot dialogues and context-sensitive task planning, leading to highly accurate conversion of natural language instructions to planned sequences of primitive actions. Moreover, we also tackle the problem of handling compound instructions containing multiple tasks.…”

Linked Data Architecture for Plan Execution in Distributed CPS

“…Plan execution has been widely studied, but many recent works focused on the architectures with single executor [13,14,27]. Munawar et al [28] proposed an architecture which considers a multi-layer architecture, but the Runtime presented in the paper does not deal with the exchange of the state updates among robots. Kootbally et al [24] proposed an architecture with direct coupling between the PDDL problem and the state of the system that is kept in a central MySQL database as well as a new Canonical Robot Command Language for executing the PDDL plans and limiting a dependency on a planning language; the possibility of using an out-of-date information is considered.…”