Grasp Pose Detection with Affordance-based Task Constraint Learning in Single-view Point Clouds

Qian, Kun; Jing, Xingshuo; Duan, Yanhui; Zhou, Bo; Fang, Fang; Xia, Jing; Ma, Xudong

doi:10.1007/s10846-020-01202-3

Cited by 35 publications

(27 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Inspired by sampling integrated with traditional affordance detection methods (Pas and Platt, 2016 ; Kanoulas et al, 2017 ; Liu C. et al, 2019 ), diverse deep learning-based affordance-based sampling techniques are proposed. Qian et al ( 2020 ) employs ResNet101 (He et al, 2017 ) with feature pyramid network (FPN) (Lin et al, 2017 ) to perform affordance detection and applies the sampling method proposed in Pas et al ( 2017 ) with refined local reference frame computation. Instead, Fang K. et al ( 2020 ) finds object affordance implicitly based on Mar et al ( 2017 ) with a multi-dimensional continuous action space and uniformly samples grasps from the object surface using antipodality constraints (Mahler et al, 2017 ).…”

Section: Grasping Candidate Generationmentioning

confidence: 99%

“…The extracted point cloud embedding is set as the input to the success probability predictor (Lu et al, 2020 ) extended by collision-free strategy (Zhou and Hauser, 2017 ; Lu and Hermans, 2019 ; Lu et al, 2020 ). Qian et al ( 2020 ) modifies fully connected layer by a novel pooling layer in R-FCN (Dai et al, 2016 ) which is able to perceive object localization change precisely. Yu Q. et al ( 2020 ) classifies grasp rectangles via a 7-layer CNN.…”

Section: Grasp Candidate Evaluationmentioning

confidence: 99%

“…Some works attempt to use object detection and instance segmentation approaches to reduce the search space in the point cloud (Lopes et al, 2018 ; Schnaubelt et al, 2019 ; Bui et al, 2020 ) or regress the grasp part using rectangle bounding boxes directly (Vohra et al, 2019 ; Yu Q. et al, 2020 ). Specifically, models designed to process point cloud furtherly accelerate the progress of candidate generation (Lin and Cong, 2019 ; Yu S. et al, 2020 ) and evaluation (Singh et al, 2018 ; Mousavian et al, 2019 ; Yan et al, 2019 ; Fang H.-S. et al, 2020 ; Qian et al, 2020 ). Moreover, end-to-end learning models mentioned in section End-to-End Learning follow the supervised learning framework as well.…”

Section: Learning Modesmentioning

confidence: 99%

“…As a result of impossibility and hardness of providing entire object observation in diverse task scenarios, single-view grasp learning plays an increasingly important role. Shape complementation (Watkins-Valls et al, 2019 ; Van der Merwe et al, 2020 ) or taking partial observation as input immediately (Yan et al, 2019 ; Qian et al, 2020 ; Qin et al, 2020 ) are tried by researchers. Even sophisticating the model design and training, single-view grasp learning enhances algorithm robustness and reduces data collection cost.…”

Section: Challenges and Future Directionsmentioning

confidence: 99%

See 3 more Smart Citations

Robotics Dexterous Grasping: The Methods Based on Point Cloud and Deep Learning

et al. 2021

View full text Add to dashboard Cite

Dexterous manipulation, especially dexterous grasping, is a primitive and crucial ability of robots that allows the implementation of performing human-like behaviors. Deploying the ability on robots enables them to assist and substitute human to accomplish more complex tasks in daily life and industrial production. A comprehensive review of the methods based on point cloud and deep learning for robotics dexterous grasping from three perspectives is given in this paper. As a new category schemes of the mainstream methods, the proposed generation-evaluation framework is the core concept of the classification. The other two classifications based on learning modes and applications are also briefly described afterwards. This review aims to afford a guideline for robotics dexterous grasping researchers and developers.

show abstract

Section: Grasping Candidate Generationmentioning

confidence: 99%

Section: Grasp Candidate Evaluationmentioning

confidence: 99%

Section: Learning Modesmentioning

confidence: 99%

Section: Challenges and Future Directionsmentioning

confidence: 99%

See 2 more Smart Citations

Robotics Dexterous Grasping: The Methods Based on Point Cloud and Deep Learning

et al. 2021

View full text Add to dashboard Cite

show abstract

“…A robot could benefit from affordance detection of parts of a single object to reduce the complexity in finding the locations of high quality grasps [30,139,139,156]. Certain objects have a clear place to be grabbed, for instance the handle of a knife, the ear of a mug or the handle of a pan.…”

Section: Grasp Point Detectionmentioning

confidence: 99%

The State of Lifelong Learning in Service Robots:

Kasaei

Melsen

Beers

et al. 2021

J Intell Robot Syst

View full text Add to dashboard Cite

Service robots are appearing more and more in our daily life. The development of service robots combines multiple fields of research, from object perception to object manipulation. The state-of-the-art continues to improve to make a proper coupling between object perception and manipulation. This coupling is necessary for service robots not only to perform various tasks in a reasonable amount of time but also to continually adapt to new environments and safely interact with non-expert human users. Nowadays, robots are able to recognize various objects, and quickly plan a collision-free trajectory to grasp a target object in predefined settings. Besides, in most of the cases, there is a reliance on large amounts of training data. Therefore, the knowledge of such robots is fixed after the training phase, and any changes in the environment require complicated, time-consuming, and expensive robot re-programming by human experts. Therefore, these approaches are still too rigid for real-life applications in unstructured environments, where a significant portion of the environment is unknown and cannot be directly sensed or controlled. In such environments, no matter how extensive the training data used for batch learning, a robot will always face new objects. Therefore, apart from batch learning, the robot should be able to continually learn about new object categories and grasp affordances from very few training examples on-site. Moreover, apart from robot self-learning, non-expert users could interactively guide the process of experience acquisition by teaching new concepts, or by correcting insufficient or erroneous concepts. In this way, the robot will constantly learn how to help humans in everyday tasks by gaining more and more experiences without the need for re-programming. In this paper, we review a set of previously published works and discuss advances in service robots from object perception to complex object manipulation and shed light on the current challenges and bottlenecks.

show abstract

Object pose estimation based on stereo vision with improvedK‐DtreeICPalgorithm

Huang

Wang

Yun

et al. 2023

Concurrency and Computation

View full text Add to dashboard Cite

SummaryWith the wide application of stereovision in SLAM, object pose estimation has gradually become one of the research hotspots. This article proposes an object pose estimation for robotic grasping based on stereo vision with improved K‐D tree ICP algorithm. The feature points and feature descriptors of the point cloud of the object to be captured are extracted, and the feature template set is established. The SAC‐IA algorithm is used to carry out initial registration of the point cloud of the target, and the ICP algorithm based on K‐D tree is used for fine registration. The experimental results show that the average coincidence degree of the final registration of the proposed object pose estimation method reaches 94.1%, and the accurate 6D pose of the object to be grasped is obtained.

show abstract

Grasp Pose Detection with Affordance-based Task Constraint Learning in Single-view Point Clouds

Cited by 35 publications

References 36 publications

Robotics Dexterous Grasping: The Methods Based on Point Cloud and Deep Learning

Robotics Dexterous Grasping: The Methods Based on Point Cloud and Deep Learning

The State of Lifelong Learning in Service Robots:

Object pose estimation based on stereo vision with improvedK‐DtreeICPalgorithm

Contact Info

Product

Resources

About