Object-Independent Human-to-Robot Handovers Using Real Time Robotic Vision

Rosenberger, Patrick; Cosgun, Akansel; Newbury, Rhys; Kwan, Jun; Ortenzi, Valerio; Corke, Peter; Grafinger, Manfred

doi:10.1109/lra.2020.3026970

Cited by 97 publications

(54 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, non-verbal communication could be used through physical properties for a human intention understanding [86]. Some other studies have taken human safety into a big consideration, developing grasping selection and reaction systems that avoid the collision with the human hand [87][88][89].…”

Section: Object Handovermentioning

confidence: 99%

Trends of Human-Robot Collaboration in Industry Contexts: Handover, Learning, and Metrics

Castro

Silva

Santos

2021

Sensors

View full text Add to dashboard Cite

Repetitive industrial tasks can be easily performed by traditional robotic systems. However, many other works require cognitive knowledge that only humans can provide. Human-Robot Collaboration (HRC) emerges as an ideal concept of co-working between a human operator and a robot, representing one of the most significant subjects for human-life improvement.The ultimate goal is to achieve physical interaction, where handing over an object plays a crucial role for an effective task accomplishment. Considerable research work had been developed in this particular field in recent years, where several solutions were already proposed. Nonetheless, some particular issues regarding Human-Robot Collaboration still hold an open path to truly important research improvements. This paper provides a literature overview, defining the HRC concept, enumerating the distinct human-robot communication channels, and discussing the physical interaction that this collaboration entails. Moreover, future challenges for a natural and intuitive collaboration are exposed: the machine must behave like a human especially in the pre-grasping/grasping phases and the handover procedure should be fluent and bidirectional, for an articulated function development. These are the focus of the near future investigation aiming to shed light on the complex combination of predictive and reactive control mechanisms promoting coordination and understanding. Following recent progress in artificial intelligence, learning exploration stand as the key element to allow the generation of coordinated actions and their shaping by experience.

show abstract

Section: Object Handovermentioning

confidence: 99%

Trends of Human-Robot Collaboration in Industry Contexts: Handover, Learning, and Metrics

Castro

Silva

Santos

2021

Sensors

View full text Add to dashboard Cite

show abstract

“…Hence, the robot directly tracks the object rather than the human hand, and no additional markers are needed, differently from the marker-based solutions proposed by Medina et al (2016) and by Pan et al (2018). Among the marker-less approaches, recent works are (Nemlekar et al, 2019;Yang et al, 2020;Rosenberger et al, 2021). Each of these works on H2R handover, and our paper, use object and/or human tracking with a different aim.…”

Section: Introductionmentioning

confidence: 99%

“…Each of these works on H2R handover, and our paper, use object and/or human tracking with a different aim. Nemlekar et al (2019) focus on reaching the handover location by tracking the human skeleton and predicting the so-called Object Transfer Point, Yang et al (2020) focus on the choice of the grasp orientation based on the human grasp type, while Rosenberger et al (2021) focus on the choice of a grasp that ensures the safety of the human using eye-in-hand vision. Our approach, instead, aims at reaching a grasp selected beforehand based on the task that the robot has to perform after the handover, i.e.…”

Section: Introductionmentioning

confidence: 99%

“…Differently from the solution by Yang et al (2020), in our work the grasp motion is not planned but directly executed in a closed-loop fashion, this allows the robot to achieve a speed comparable to the human one while automatically reacting to the giver motion. The approach proposed by Rosenberger et al (2021) is computationally demanding and it is still not suitable for real-time closed-loop control. In contrast, our approach requires an object database containing the grasp poses (this is a fair assumption in a supermarket scenario) but enables a fast closed-loop control and can track the object even if the human changes the object pose during the handover execution.…”

Section: Introductionmentioning

confidence: 99%

“…In contrast, our approach requires an object database containing the grasp poses (this is a fair assumption in a supermarket scenario) but enables a fast closed-loop control and can track the object even if the human changes the object pose during the handover execution. The main limit of the solution by Nemlekar et al (2019) is the need to keep the entire skeleton of the human body within the field of view of the camera to allow the tracking algorithm to work properly; this limit has been overcome by Rosenberger et al (2021) who can track only parts of the human body as well as our approach which tracks the object only.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Handover Control for Human-Robot and Robot-Robot Collaboration

2021

View full text Add to dashboard Cite

Modern scenarios in robotics involve human-robot collaboration or robot-robot cooperation in unstructured environments. In human-robot collaboration, the objective is to relieve humans from repetitive and wearing tasks. This is the case of a retail store, where the robot could help a clerk to refill a shelf or an elderly customer to pick an item from an uncomfortable location. In robot-robot cooperation, automated logistics scenarios, such as warehouses, distribution centers and supermarkets, often require repetitive and sequential pick and place tasks that can be executed more efficiently by exchanging objects between robots, provided that they are endowed with object handover ability. Use of a robot for passing objects is justified only if the handover operation is sufficiently intuitive for the involved humans, fluid and natural, with a speed comparable to that typical of a human-human object exchange. The approach proposed in this paper strongly relies on visual and haptic perception combined with suitable algorithms for controlling both robot motion, to allow the robot to adapt to human behavior, and grip force, to ensure a safe handover. The control strategy combines model-based reactive control methods with an event-driven state machine encoding a human-inspired behavior during a handover task, which involves both linear and torsional loads, without requiring explicit learning from human demonstration. Experiments in a supermarket-like environment with humans and robots communicating only through haptic cues demonstrate the relevance of force/tactile feedback in accomplishing handover operations in a collaborative task.

show abstract

Multimodal Learning‐Based Proactive Human Handover Intention Prediction Using Wearable Data Gloves and Augmented Reality

Zou,

Liu,

Zhao

et al. 2024

Advanced Intelligent Systems

View full text Add to dashboard Cite

Efficient object handover between humans and robots holds significant importance within collaborative manufacturing environments. Enhancing the efficacy of human–robot handovers involves enabling robots to comprehend and foresee human handover intentions. This article introduces human‐teaching–robot‐learning‐prediction framework, allowing robots to learn from diverse human demonstrations and anticipate human handover intentions. The framework facilitates human programming of robots through demonstrations utilizing augmented reality and a wearable dataglove, aligned with task requirements and human working preferences. Subsequently, robots enhance their cognitive capabilities by assimilating insights from human handover demonstrations, utilizing deep neural network algorithms. Furthermore, robots can proactively seek clarification from humans via an augmented reality system when confronted with ambiguity in human intentions, mirroring how humans seek clarity from their counterparts. This proactive approach empowers robots to anticipate human intentions and assist human partners during handovers. Empirical results underscore the benefits of the proposed approach, demonstrating highly accurate prediction of human intentions in human–robot handover tasks.

show abstract

Object-Independent Human-to-Robot Handovers Using Real Time Robotic Vision

Cited by 97 publications

References 26 publications

Trends of Human-Robot Collaboration in Industry Contexts: Handover, Learning, and Metrics

Trends of Human-Robot Collaboration in Industry Contexts: Handover, Learning, and Metrics

Handover Control for Human-Robot and Robot-Robot Collaboration

Multimodal Learning‐Based Proactive Human Handover Intention Prediction Using Wearable Data Gloves and Augmented Reality

Contact Info

Product

Resources

About