Task Planning with a Weighted Functional Object-Oriented Network

Abstract

Following work on joint object-action representations, functional object-oriented networks (FOON) were introduced as a knowledge graph representation for robots. Taking the form of a bipartite graph, a FOON contains symbolic (high-level) concepts useful to a robot's understanding of tasks and its environment for object-level planning. Prior to this paper, little has been done to demonstrate how task plans acquired from FOON via task tree retrieval can be executed by a robot, as the concepts in a FOON are too abstract for immediate execution. We propose a hierarchical task planning approach that translates a FOON graph into a PDDL-based representation of domain knowledge for manipulation planning. As a result of this process, a task plan can be acquired that a robot can execute from start to end, leveraging the use of action contexts and skills in the form of dynamic movement primitives (DMP). We demonstrate the entire pipeline from planning to execution using CoppeliaSim and show how learned action contexts can be extended to never-before-seen scenarios.

Functional Task Tree Generation from a Knowledge Graph to Solve Unseen Problems

“…In a typically large FOON, exploring all paths can be time-consuming and memory inefficient. Hence, unlike the previous search algorithm [25], we choose a sub-optimal Algorithm 3 retrieve reference task tree Input: Desired object G and required ingredients I 1: Let Q be the queue of nodes to search, S be the list of tree nodes, and R be the list of root nodes 2: R ← Find all functional units FU where G is an output 3: Q.push(R) 4: while Q is not empty do 5:…”

“…The algorithm draws upon the depth-first search (DFS) and breadth-first search (BFS) algorithms: starting from the goal node, we search for candidate functional units in a depth-wise manner, while for each candidate unit, we search among its input nodes in a breadth-wise manner to determine whether or not they are in the kitchen. In [25], we introduced an alternative retrieval algorithm that explores different paths to achieving a goal; this was used to evaluate the most optimal path with a weighted FOON, where each functional unit is assigned a weight corresponding to its success rate of execution. This version of the algorithm is used to derive task trees for our task tree generation procedure.…”

“…P ← Take Cartesian Product from E {Refer to [25] for the detail of tree node creation} T ← Choose path that has maximum score 21: end for Output: T approach where some branches are pruned based on the ingredients stored at that node and their relevance with I.…”

“…We compute the Cartesian Product P of units in C to generate valid paths with different combination of functional units. We refer the interested reader to [25] for the details of creating a tree S from P . The search continues until all dependencies are met.…”

“…Each path is a possible solution to reach the goal. As opposed to the procedure in [25], where a path is selected that maximizes robot's success rate, we select the path with the most ingredient overlap with I. If there are multiple paths with similar overlap, we select the shortest path, i.e., the one with minimum number of functional units.…”