Hallucinating Humans for Learning Robotic Placement of Objects

Abstract

Abstract. While a significant body of work has been done on grasping objects, there is little prior work on placing and arranging objects in the environment. In this work, we consider placing multiple objects in complex placing areas, where neither the object nor the placing area may have been seen by the robot before. Specifically, the placements should not only be stable, but should also follow human usage preferences. We present learning and inference algorithms that consider these aspects in placing. In detail, given a set of 3D scenes containing objects, our method, based on Dirichlet process mixture models, samples human poses in each scene and learns how objects relate to those human poses. Then given a new room, our algorithm is able to select meaningful human poses and use them to determine where to place new objects. We evaluate our approach on a variety of scenes in simulation, as well as on robotic experiments.

A geometrical placement planner for unknown sensor-modelled objects and placement areas

“…One of the first overall solutions for placing novel objects in complex placement areas is introduced by Jiang et al [18] and [19]. They outline a learning based framework that requires an object database together with semantic labels.…”

Modeling High-Dimensional Humans for Activity Anticipation using Gaussian Process Latent CRFs

“…Grabner et al [6] and Gupta et al [8] utilize imaginary human actors to detect objects and human workspace in 2D images. Jiang et al [13,11,12] apply hallucinated human configurations to a robotic task of arranging 3D indoor scenes. In another previous work of anticipating human activities [20,19], human motions are implicitly modeled through object trajectories.…”

Hallucinated Humans as the Hidden Context for Labeling 3D Scenes

“…Jiang, Lim and Saxena [14,17] used hallucinated humans for learning the object arrangements in a house in order to enable robots to place objects in human-preferred locations. However, they assumed that the objects have been detected.…”

“…The concept of affordances, proposed by Gibson [7], has recently become the focus of many works in cognitive vision (e.g., [24]) and robotics [14,17]. Grabner et al [8] apply this idea to detect the functionality of the object (specifically, chairs), and then combine this information with visual object appearance to perform object classification.…”