Generalizing Object-Centric Task-Axes Controllers using Keypoints

Abstract

To perform manipulation tasks in the real world, robots need to operate on objects with various shapes, sizes and without access to geometric models. It is often unfeasible to train monolithic neural network policies across such large variance in object properties. Towards this generalization challenge, we propose to learn modular task policies which compose object-centric task-axes controllers. These task-axes controllers are parameterized by properties associated with underlying objects in the scene. We infer these controller parameters directly from visual input using multi-view dense correspondence learning. Our overall approach provides a simple and yet powerful framework for learning manipulation tasks. We empirically evaluate our approach on 3 different manipulation tasks and show its ability to generalize to large variance in object size, shape and geometry.. . .

SpaTiaL: monitoring and planning of robotic tasks using spatio-temporal logic specifications

“…These spatial relations help robots to understand desired goal configurations of objects (Izatt and Tedrake , 2020) and to predict an action's outcome (Paus et al, 2020). To this end, many approaches perform object-centric planning (Devin et al, 2018;Sharma et al, 2020;Shridhar et al, 2022), where planning is performed in the task space of the object itself (Manuelli et al, 2019;Migimatsu & Bohg, 2020;Agostini and Lee, 2020).…”