Learning to grasp familiar objects using object view recognition and template matching

Shafii, Nima; Kasaei, S. Hamidreza; Lopes, Luís Seabra

doi:10.1109/iros.2016.7759448

Cited by 17 publications

(15 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For service robots which need to be able to help those in need, significant chance of failure to recognise and grasp objects, for important objects like medicine containers, could be very problematic. Towards addressing this issue, some researchers used kinesthetic teaching to teach a new grasp configuration, including the position and orientation of the arm relative to the object and the finger positions [67,90,172]. As shown in Fig.…”

Section: Open-ended Grasp Learningmentioning

confidence: 99%

The State of Lifelong Learning in Service Robots:

Kasaei

Melsen

Beers

et al. 2021

J Intell Robot Syst

Self Cite

View full text Add to dashboard Cite

Service robots are appearing more and more in our daily life. The development of service robots combines multiple fields of research, from object perception to object manipulation. The state-of-the-art continues to improve to make a proper coupling between object perception and manipulation. This coupling is necessary for service robots not only to perform various tasks in a reasonable amount of time but also to continually adapt to new environments and safely interact with non-expert human users. Nowadays, robots are able to recognize various objects, and quickly plan a collision-free trajectory to grasp a target object in predefined settings. Besides, in most of the cases, there is a reliance on large amounts of training data. Therefore, the knowledge of such robots is fixed after the training phase, and any changes in the environment require complicated, time-consuming, and expensive robot re-programming by human experts. Therefore, these approaches are still too rigid for real-life applications in unstructured environments, where a significant portion of the environment is unknown and cannot be directly sensed or controlled. In such environments, no matter how extensive the training data used for batch learning, a robot will always face new objects. Therefore, apart from batch learning, the robot should be able to continually learn about new object categories and grasp affordances from very few training examples on-site. Moreover, apart from robot self-learning, non-expert users could interactively guide the process of experience acquisition by teaching new concepts, or by correcting insufficient or erroneous concepts. In this way, the robot will constantly learn how to help humans in everyday tasks by gaining more and more experiences without the need for re-programming. In this paper, we review a set of previously published works and discuss advances in service robots from object perception to complex object manipulation and shed light on the current challenges and bottlenecks.

show abstract

Section: Open-ended Grasp Learningmentioning

confidence: 99%

The State of Lifelong Learning in Service Robots:

Kasaei

Melsen

Beers

et al. 2021

J Intell Robot Syst

Self Cite

View full text Add to dashboard Cite

show abstract

“…In previous work, we adopted an approach based on 3D partial object views. The target object view was represented by bagof-words and the grasp was represented by the local shape of the object around the grasp point and a global feature of the grasp point [25]. There was no clear separation between the object category and the grasp affordance.…”

Section: Related Workmentioning

confidence: 99%

“…One of the main challenges is to decide which visual cues should be used as features of the taught grasp region. Following previous work [25], a combination of a local shape feature (a spin-image [33]) and a simple global feature is used. Towards this end, a key-point in the grasp region is selected based on the grasp line, i.e., a line defined by the orientation of the end-effector and passing in its center.…”

Section: Grasp Learning and Detectionmentioning

confidence: 99%

Interactive Open-Ended Object, Affordance and Grasp Learning for Robotic Manipulation

Kasaei

Shafii

Lopes

et al. 2019

2019 International Conference on Robotics and Automation (ICRA)

Self Cite

View full text Add to dashboard Cite

Service robots are expected to autonomously and efficiently work in human-centric environments. For this type of robots, object perception and manipulation are challenging tasks due to need for accurate and real-time response. This paper presents an interactive open-ended learning approach to recognize multiple objects and their grasp affordances concurrently. This is an important contribution in the field of service robots since no matter how extensive the training data used for batch learning, a robot might always be confronted with an unknown object when operating in human-centric environments. The paper describes the system architecture and the learning and recognition capabilities. Grasp learning associates grasp configurations (i.e., end-effector positions and orientations) to grasp affordance categories. The grasp affordance category and the grasp configuration are taught through verbal and kinesthetic teaching, respectively. A Bayesian approach is adopted for learning and recognition of object categories and an instance-based approach is used for learning and recognition of affordance categories. An extensive set of experiments has been performed to assess the performance of the proposed approach regarding recognition accuracy, scalability and grasp success rate on challenging datasets and real-world scenarios.All authors are with IEETA -University of Aveiro, 3810-193, Aveiro, Portugal. S. Hamidreza Kasaei is also with

show abstract

“…Model-based and learning-based methods could be used to plan grasps across a wide variety of objects. Existing physical grasp analysis techniques, such as grasp quality metrics [2], template matching [3], and wrench space analysis [4], can be used to search for the optimal grasp. These approaches, however, can be less robust in practice due to the perception limitation.…”

Section: Introductionmentioning

confidence: 99%

6-DoF Contrastive Grasp Proposal Network

Zhu

Sun

Yi³

et al. 2021

2021 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Proposing grasp poses for novel objects is an essential component for any robot manipulation task. Planning six degrees of freedom (DoF) grasps with a single camera, however, is challenging due to the complex object shape, incomplete object information, and sensor noise. In this paper, we present a 6-DoF contrastive grasp proposal network (CGPN) to infer 6-DoF grasps from a single-view depth image. First, an image encoder is used to extract the feature map from the input depth image, after which 3-DoF grasp regions are proposed from the feature map with a rotated region proposal network. Feature vectors that within the proposed grasp regions are then extracted and refined to 6-DoF grasps. The proposed model is trained offline with synthetic grasp data. To improve the robustness in reality and bridge the simulation-to-real gap, we further introduce a contrastive learning module and variant image processing techniques during the training. CGPN can locate collision-free grasps of an object using a single-view depth image within 0.5 second. Experiments on a physical robot further demonstrate the effectiveness of the algorithm. The experimental videos are available at [1].

show abstract

Learning to grasp familiar objects using object view recognition and template matching

Cited by 17 publications

References 13 publications

The State of Lifelong Learning in Service Robots:

The State of Lifelong Learning in Service Robots:

Interactive Open-Ended Object, Affordance and Grasp Learning for Robotic Manipulation

6-DoF Contrastive Grasp Proposal Network

Contact Info

Product

Resources

About