Target-Mass Grasping of Entangled Food Using Pre-Grasping &amp; Post-Grasping

Takahashi, Kuniyuki; Fukaya, Naoki; Ummadisingu, Avinash

doi:10.1109/lra.2021.3138553

Cited by 10 publications

(3 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Ray et al [40] proposed a method using a designed two-finger gripper to untangle herbs from a pile. Takahashi et al [41] developed a learning-based separation strategy for grasping a specified mass of small food pieces. Although studies has addressed factory automation problems related to wire harnesses [22], [23], [24], robotic bin picking using entangled wire harness remains relatively unexplored.…”

Section: Manipulation Of Multiple Deformable Objectsmentioning

confidence: 99%

Learning Efficient Policies for Picking Entangled Wire Harnesses: An Approach to Industrial Bin Picking

Zhang

Domae

Wan

et al. 2023

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Industrial bin picking for tangled-prone objects requires the robot to either pick up untangled objects or perform separation manipulation when the bin contains no isolated objects. The robot must be able to flexibly perform appropriate actions based on the current observation. It is challenging due to high occlusion in the clutter, elusive entanglement phenomena, and the need for skilled manipulation planning. In this paper, we propose an autonomous, effective and general approach for picking up tangled-prone objects for industrial bin picking. First, we learn PickNet -a network that maps the visual observation to pixel-wise possibilities of picking isolated objects or separating tangled objects and infers the corresponding grasp. Then, we propose two effective separation strategies: Dropping the entangled objects into a buffer bin to reduce the degree of entanglement; Pulling to separate the entangled objects in the buffer bin planned by PullNet -a network that predicts position and direction for pulling from visual input. To efficiently collect data for training PickNet and PullNet, we embrace the self-supervised learning paradigm using an algorithmic supervisor in a physics simulator. Realworld experiments show that our policy can dexterously pick up tangled-prone objects with success rates of 90%. We further demonstrate the generalization of our policy by picking a set of unseen objects. Supplementary material, code, and videos can be found at https://xinyiz0931.github.io/tangle.

show abstract

Section: Manipulation Of Multiple Deformable Objectsmentioning

confidence: 99%

Learning Efficient Policies for Picking Entangled Wire Harnesses: An Approach to Industrial Bin Picking

Zhang

Domae

Wan

et al. 2023

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

show abstract

“…Matsuo et al developed an AI robot [6] hand that can grasp shredded cabbage quantitatively using reinforcement learning. Two hundred training cycles in 2 hours resulted in a grasping error of 4.88 g on average against a target value of 50 g. Takahashi et al defined entangled foods, such as shredded cabbage and Japan sakuragi.t.aa at m.titech.ac.jp bean sprouts, as foods that are difficult to grasp because they become entangled and lumpy, and attempted quantitative grasping [7]. By performing specific actions before and after grasping, they were able to reduce entanglement and adjust the amount of food to be grasped, and by using an RGB-D camera to photograph the tray from the top, they were able to estimate the gripper drop position at which the optimal amount of food could be grasped, and succeeded in significantly improving grasping accuracy.…”

Section: Introductionmentioning

confidence: 99%

Chobit Hand: A Gripper for A Linear-Shaped Food Adjusting Precise Weight of Serving

Sakuragi

Nabae

Suzumori

et al. 2023

2023 IEEE/SICE International Symposium on System Integration (SII)

View full text Add to dashboard Cite

In many cases in the field of food processing and production, serving is still done by workers rather than by robots. However, linear foods such as spaghetti need to be served precisely and quantitatively, which is difficult for a conventional robot gripper. Therefore, in this study, we developed a new end-effector that can grasp a small amount of linear food with high accuracy. By installing several fingers on this end-effector and driving a fixed number of fingers according to the error between the target and the measured weight of food, we can compensate for the error. After investigating several parameters and selecting the optimal finger shape, the end-effector grasped one noodle with a success rate of nearly 90%. Next, we introduced capacitance sensors to all fingers to detect successful grasping. The sensors enabled us to construct a system that compensates for the weight errors in the amount of serving by detecting whether each finger is grasping and controlling the number of fingers that open accordingly. We conducted experiments with spaghetti and succeeded in adjusting the amount of food with an error of less than 2g.

show abstract

“…In addition, foods are fragile and easily damaged by interaction and change in behavior over time due to environmental factors (temperature, humidity) as well as loss of water content and spoilage. It makes the gathering of large datasets of real food challenging and often requires techniques to learn from limited samples [2], [3].…”

Section: Introductionmentioning

confidence: 99%

Cluttered Food Grasping with Adaptive Fingers and Synthetic-Data Trained Object Detection

Ummadisingu¹,

Takahashi²,

Fukaya³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

The food packaging industry handles an immense variety of food products with wide-ranging shapes and sizes, even within one kind of food. Menus are also diverse and change frequently, making automation of pick-and-place difficult. A popular approach to bin-picking is to first identify each piece of food in the tray by using an instance segmentation method. However, human annotations to train these methods are unreliable and error-prone since foods are packed close together with unclear boundaries and visual similarity making separation of pieces difficult. To address this problem, we propose a method that trains purely on synthetic data and successfully transfers to the real world using sim2real methods by creating datasets of filled food trays using high-quality 3d models of real pieces of food for the training instance segmentation models. Another concern is that foods are easily damaged during grasping. We address this by introducing two additional methods-a novel adaptive finger mechanism to passively retract when a collision occurs, and a method to filter grasps that are likely to cause damage to neighbouring pieces of food during a grasp. We demonstrate the effectiveness of the proposed method on several kinds of real foods.

show abstract

Target-Mass Grasping of Entangled Food Using Pre-Grasping & Post-Grasping

Cited by 10 publications

References 16 publications

Learning Efficient Policies for Picking Entangled Wire Harnesses: An Approach to Industrial Bin Picking

Learning Efficient Policies for Picking Entangled Wire Harnesses: An Approach to Industrial Bin Picking

Chobit Hand: A Gripper for A Linear-Shaped Food Adjusting Precise Weight of Serving

Cluttered Food Grasping with Adaptive Fingers and Synthetic-Data Trained Object Detection

Contact Info

Product

Resources

About