Data-driven robotic visual grasping detection for unknown objects: A problem-oriented review

Tian, Hongkun; Song, Kechen; Li, Song; Ma, Shuai; Xu, Jing; Yan, Yunhui

doi:10.1016/j.eswa.2022.118624

Cited by 33 publications

(8 citation statements)

References 92 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Among these, grippers capable of grasping small objects in cluttered environments often focus on specific usage or are limited to two parallel grippers. Even simple grippers typically require grasp detection and the prediction of the grasp pose to handle unknown objects using external vision [18]. However, objects are challenging to grasp in cluttered environments, and the environment constantly changes as the gripper interacts with it.…”

Section: Methodsmentioning

confidence: 99%

DenseTact: Optical Tactile Sensor for Dense Shape Reconstruction

Do¹,

Kennedy²

2022

2022 International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Collaborative robots stand to have an immense impact both on human welfare in domestic service applications and industrial superiority in advanced manufacturing requires dexterous assembly. The outstanding challenge is providing robotic fingertips with a physical design that makes them adept at performing dexterous tasks that require high-resolution, calibrated shape reconstruction and force sensing. In this work, we present DenseTact 2.0, an optical-tactile sensor capable of visualizing the deformed surface of a soft fingertip and using that image in a neural network to perform both calibrated shape reconstruction and 6-axis wrench estimation. We demonstrate the sensor accuracy of 0.3633mm per pixel for shape reconstruction, 0.410N for forces, 0.387mmN •m for torques, and the ability to calibrate new fingers through transfer learning, achieving comparable performance that trained more than four times faster with only 12% of the dataset size.

show abstract

Section: Methodsmentioning

confidence: 99%

DenseTact: Optical Tactile Sensor for Dense Shape Reconstruction

Do¹,

Kennedy²

2022

2022 International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

show abstract

“…Numerous recent works focus on the dexterous manipulation of unknown objects, where "unknown object" refers to an object whose shape and other properties are unknown when the manipulation begins [8]. The application of force optimization for the manipulation of unknown objects is presented in [14], and online improvement of the hand movements while manipulating an object is described in [15][16][17]. The use of virtual contact points to solve the dexterous manipulation of unknown objects is presented in [8], and the teleoperated dexterous manipulation by an operator using an input interface, such as a keyboard, is carried out in [18], where the task is carried out by the hand following the directions of the operator.…”

Section: Introductionmentioning

confidence: 99%

Learning-Based Planner for Unknown Object Dexterous Manipulation Using ANFIS

Sheikhsamad,

Suárez,

Rosell

2024

Machines

View full text Add to dashboard Cite

Dexterous manipulation of unknown objects performed by robots equipped with mechanical hands represents a critical challenge. The difficulties arise from the absence of a precise model of the manipulated objects, unpredictable environments, and limited sensing capabilities of the mechanical hands compared to human hands. This paper introduces a data-driven approach that provides a learning-based planner for dexterous manipulation employing an Adaptive Neuro-Fuzzy Inference System (ANFIS) fed by data obtained from an analytical manipulation planner. ANFIS captures the complex relationships between inputs and optimal manipulation parameters. Moreover, during a training phase, it is able to fine-tune itself on the basis of its experiences. The proposed planner enables a robot to interact with objects of various shapes, sizes, and material properties while providing an adaptive solution for increasing robotic dexterity. The planner is validated in a real-world environment, applying an Allegro anthropomorphic robotic hand. A link to a video of the experiment is provided in the paper.

show abstract

“…With the continuous advancement of industrial robot automation production lines, machine vision and artificial intelligence technology have been widely used in industrial robots, especially for robot identification classification and object pickup work [1,2,3]. Parallel robots play an important role in food packaging, workpiece sorting, and intelligent assembly due to their good structural stability and fast pickup characteristics [4,5].…”

Section: Introductionmentioning

confidence: 99%

A 3D pickup estimation method for industrial parallel robots based on point cloud simplification and registration

Zhang,

Wang,

Wang

et al. 2023

Preprint

View full text Add to dashboard Cite

In industrial environments, cameras are strongly affected by light and viewpoints, and parallel robots based on traditional vision methods have poor sorting abilities. In two-dimensional vision, depth information cannot be estimated, and parallel robots can only pick up objects based on the fixed height. In this paper, we propose a 3D pickup estimation method for parallel robots based on point cloud simplification and registration for multi-objective pickup tasks. Firstly, a point cloud segmentation method based on the spatial domain is proposed to separate incomplete object point clouds from robot space. The 3D scanner scans the complete object point clouds to produce the object dataset. Secondly, according to the fast point feature histogram (FPFH) and the weight locally optimal projection (WLOP), a fusing the FPFH and WLOP (FF-WLOP) method is proposed to simplify the incomplete point cloud and obtain more distinctive edge features of objects. The complete point cloud in the dataset is aligned with the simplified incomplete point cloud and the coordinate information of the calculated barycenter is given to the incomplete point cloud. Then a dynamic weight singular value decomposition (D-SVD) hand-eye calibration method and a best projection point strategy are proposed to transform the barycenter coordinates of the object to the best pickup coordinates. The experiment results show that the point cloud registration error is 0.38mm, the pickup rate is 92%, and the robot positioning error is 4.67mm, which meets the basic pickup requirements.

show abstract

Data-driven robotic visual grasping detection for unknown objects: A problem-oriented review

Cited by 33 publications

References 92 publications

DenseTact: Optical Tactile Sensor for Dense Shape Reconstruction

DenseTact: Optical Tactile Sensor for Dense Shape Reconstruction

Learning-Based Planner for Unknown Object Dexterous Manipulation Using ANFIS

A 3D pickup estimation method for industrial parallel robots based on point cloud simplification and registration

Contact Info

Product

Resources

About