Put the Bear on the Chair! Intelligent Robot Interaction with Previously Unseen Chairs via Robot Imagination

Abstract

In this letter, we study the problem of autonomously placing a teddy bear on a previously unseen chair for sitting. To achieve this goal, we present a novel method for robots to imagine the sitting pose of the bear by physically simulating a virtual humanoid agent sitting on the chair. We also develop a robotic system which leverages motion planning to plan SE(2) motions for a humanoid robot to walk to the chair and whole-body motions to put the bear on it, respectively. Furthermore, to cope with the cases where the chair is not in an accessible pose for placing the bear, a human-robot interaction (HRI) framework is introduced in which a human follows language instructions given by the robot to rotate the chair and help make the chair accessible. We implement our method with a robot arm and a humanoid robot. We calibrate the proposed system with 3 chairs and test on 12 previously unseen chairs in both accessible and inaccessible poses extensively. Results show that our method enables the robot to autonomously put the teddy bear on the 12 unseen chairs with a very high success rate. The HRI framework is also shown to be very effective in changing the accessibility of the chair. Source code will be available. Video demos are available at https: //chirikjianlab.github.io/putbearonchair/.

Prepare the Chair for the Bear! Robot Imagination of Sitting Affordance to Reorient Previously Unseen Chairs

“…Here we revisit the imagination method introduced in [3], [4] briefly. The human agent is modeled with an articulated human body with the forearms and feet trimmed off [26], [27].…”

“…For each drop, the agent resultant configuration C res is evaluated by the sitting affordance model (SAM) developed from [3], [4] with five criteria. The first two criteria are similar to that in [3], [4]. 1) Joint Angle.…”

“…In our setting, an upright chair is accessible if it is facing the NAO robot and inaccessible if otherwise (i.e., the chair faces the robot arm or the longer edge of the table). The NAO uses the same method in [4] to plan the SE(2) trajectory to walk to the chair and the whole-body motion to seat the bear. In the experiments, for a functional pose, our approach considers it inaccessible if the planning fails.…”

“…In this case, the robot needs to further rotate the object to a functional and accessible pose that is sittable for the bear if necessary. In particular, we extend the scope of [3] and [4] by leveraging the imagination method to guide the manipulation for object reconstruction and the manipulation for preparing the chair to a functional and accessible pose. Fig.…”

“…No real robot experiments are performed. In [4], the focus lies on finding the agent sitting pose for an object known as a chair in an upright pose a priori. Thus, no robot manipulation of the chair is included.…”