Learning Dense Visual Correspondences in Simulation to Smooth and Fold Real Fabrics

Ganapathi, Aditya; Sundaresan, Priya; Thananjeyan, Brijen; Seita, Daniel; Grannen, Jennifer; Hwang, Mina; Hoque, Ryan; Jamali, Nawid; Yamane, Katsu; Iba, Soshi; Goldberg, Ken

doi:10.1109/icra48506.2021.9561980

Cited by 46 publications

(32 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Florence et al [4] propose mapping object images to a dense pixel-wise descriptor embedding with which to recover object pose in both single and cluttered multi-instance scenes for semantic grasping. Dense descriptors have proven effective in rope knot-tying [23] and cloth folding and smoothing [5], but global descriptors lack robustness to severe deformation and occlusion, as encountered with multiple overlapping cables.…”

Section: A Deformable Object Manipulationmentioning

confidence: 99%

Disentangling Dense Multi-Cable Knots

Viswanath¹,

Grannen²,

Sundaresan³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Disentangling two or more cables requires many steps to remove crossings between and within cables. We formalize the problem of disentangling multiple cables and present an algorithm, Iterative Reduction Of Non-planar Multiple cAble kNots (IRON-MAN), that outputs robot actions to remove crossings from multi-cable knotted structures. We instantiate this algorithm with a learned perception system, inspired by prior work in single-cable untying that given an image input, can disentangle two-cable twists, three-cable braids, and knots of two or three cables, such as overhand, square, carrick bend, sheet bend, crown, and fisherman's knots. IRON-MAN keeps track of task-relevant keypoints corresponding to target cable endpoints and crossings and iteratively disentangles the cables by identifying and undoing crossings that are critical to knot structure. Using a da Vinci surgical robot, we experimentally evaluate the effectiveness of IRON-MAN on untangling multicable knots of types that appear in the training data, as well as generalizing to novel classes of multi-cable knots. Results suggest that IRON-MAN is effective in disentangling knots involving up to three cables with 80.5% success and generalizing to knot types that are not present during training, with cables of both distinct or identical colors.

show abstract

Section: A Deformable Object Manipulationmentioning

confidence: 99%

Disentangling Dense Multi-Cable Knots

Viswanath¹,

Grannen²,

Sundaresan³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Work investigating robotic cloth manipulation has considered tasks that involve smoothing, folding, or unfolding cloth using a variety of task-oriented and robot learning-based approaches [2], [3], [4]. Task-oriented approaches generally use traditional perception-based algorithms to identify manually selected cloth features, like hems, corners, or wrinkles [5], [6], [7], [8], [9], [10], [11].…”

Section: Related Work a Bedding And Cloth Manipulationmentioning

confidence: 99%

“…While learning-based cloth manipulation approaches in the real world have yielded promising results for a variety of tasks, the potential for generalization is limited due to the excessive cost associated with collecting a training data set of sufficiently large size [3], [15]. Several recent methods have leveraged simulation as a way to learn general manipulation skills of cloth, such as folding, yet do not consider the manipulation of cloth around people [16], [17], [18].…”

Section: Related Work a Bedding And Cloth Manipulationmentioning

confidence: 99%

Bodies Uncovered: Learning to Manipulate Real Blankets Around People via Physics Simulations

Puthuveetil¹,

Kemp²,

Erickson³

2021

Preprint

View full text Add to dashboard Cite

While robots present an opportunity to provide physical assistance to older adults and people with mobility impairments in bed, people frequently rest in bed with blankets that cover the majority of their body. To provide assistance for many daily self-care tasks, such as bathing, dressing, or ambulating, a caregiver must first uncover blankets from part of a person's body. In this work, we introduce a formulation for robotic bedding manipulation around people in which a robot uncovers a blanket from a target body part while ensuring the rest of the human body remains covered. We compare both reinforcement and supervised learning approaches for optimizing policies which provide a robot with grasp and release points that uncover a target part of the body. We trained and conducted evaluations of these policies in physics simulation environments that consist of a deformable cloth mesh covering a simulated human lying supine on a bed. In addition, we transfer simulation-trained policies to a real mobile manipulator and demonstrate that it can uncover a blanket from target body parts of a manikin lying in bed. Source code is available online 1 .

show abstract

“…Imitation learning (IL) [1,2,3] has seen success in a variety of robotic tasks ranging from autonomous driving [4,5,6] to robotic manipulation [7,8,9,10,11]. In its simplest form, the human provides an offline set of task demonstrations to the robot, which the robot uses to match human behavior.…”

Section: Introductionmentioning

confidence: 99%

ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learning

Hoque,

Balakrishna,

Novoseller

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Effective robot learning often requires online human feedback and interventions that can cost significant human time, giving rise to the central challenge in interactive imitation learning: is it possible to control the timing and length of interventions to both facilitate learning and limit burden on the human supervisor? This paper presents ThriftyDAgger, an algorithm for actively querying a human supervisor given a desired budget of human interventions. ThriftyDAgger uses a learned switching policy to solicit interventions only at states that are sufficiently (1) novel, where the robot policy has no reference behavior to imitate, or (2) risky, where the robot has low confidence in task completion. To detect the latter, we introduce a novel metric for estimating risk under the current robot policy. Experiments in simulation and on a physical cable routing experiment suggest that ThriftyDAgger's intervention criteria balances task performance and supervisor burden more effectively than prior algorithms. ThriftyDAgger can also be applied at execution time, where it achieves a 100% success rate on both the simulation and physical tasks. A user study (N = 10) in which users control a three-robot fleet while also performing a concentration task suggests that ThriftyDAgger increases human and robot performance by 58% and 80% respectively compared to the next best algorithm while reducing supervisor burden. See https://tinyurl.com/thrifty-dagger for supplementary material.

show abstract

Learning Dense Visual Correspondences in Simulation to Smooth and Fold Real Fabrics

Cited by 46 publications

References 31 publications

Disentangling Dense Multi-Cable Knots

Disentangling Dense Multi-Cable Knots

Bodies Uncovered: Learning to Manipulate Real Blankets Around People via Physics Simulations

ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learning

Contact Info

Product

Resources

About