Benchmark for Human-to-Robot Handovers of Unseen Containers With Unknown Filling

Abstract

The real-time estimation through vision of the physical properties of objects manipulated by humans is important to inform the control of robots for performing accurate and safe grasps of objects handed over by humans. However, estimating the 3D pose and dimensions of previously unseen objects using only RGB cameras is challenging due to illumination variations, reflective surfaces, transparencies, and occlusions caused both by the human and the robot. In this letter, we present a benchmark for dynamic human-to-robot handovers that do not rely on a motion capture system, markers, or prior knowledge of specific objects. To facilitate comparisons, the benchmark focuses on cups with different levels of transparencies and with an unknown amount of an unknown filling. The performance scores assess the overall system as well as its components in order to help isolate modules of the pipeline that need improvements. In addition to the task description and the performance scores, we also present and distribute as open source a baseline implementation for the overall pipeline to enable comparisons and facilitate progress.

Trends of Human-Robot Collaboration in Industry Contexts: Handover, Learning, and Metrics

“…There are many concrete applications, where a robot has proven to be useful in helping humans by handing over objects, such as short-cycle repetitive tasks [67] or dynamic object handover tasks [68,69]. In the last decade, some studies described the process of object handover between humans with the purpose of transferring that same execution to robots [70,71].…”

“…Bi-directional [67,72] Fluent Movements (Human-like) [68,72,76,83,84] Learning [68,74,77,[93][94][95] Give/Receive Moving Object [68,69] Object Possession decision contact point…”

“…As mentioned previously, existing research includes context-aware object recognition [69], gesture recognition [17], human patterns recognition [99], human action prediction [98], learning manipulation skills from human demonstrations [103], and fine-tuning learned skills by exploring efficient forms of reinforcement learning [106]. Machine learning methods are increasingly used in robotic applications due to advances in their mathematical formalization.…”

Guest Editorial: Introduction to the Special Issue on Benchmarking Protocols for Robotic Manipulation

“…In [12], a benchmark is provided for assessing aerial manipulation capabilities, e.g., aerial grasping, force control etc. The work in [13] focuses on dynamic human-to-robot handovers, and provides experimental procedures for transferring cups with different fullness levels and transparencies. The work in [14] provides a framework to assess the performance of robotic manipulation simulators via a real-world dataset.…”

From Movement Kinematics to Object Properties: Online Recognition of Human Carefulness

“…Let us imagine a robot which is asked to receive a glass of water from a human: it should recognize the human carefulness to manipulate the glass without spilling water. The carefulness has been explored in studies of human-human handovers to teach robots how to correctly transfer objects [12,3], monitoring human movements with motion capture sensors. In a previous study, we demonstrated that it is possible to train a classifier to distinguish between careful and non careful human motions using only data from a low-resolution camera [7].…”