A Mobile Manipulation System for One-Shot Teaching of Complex Tasks in Homes

Bajracharya, Max; Borders, James; Helmick, Dan; Kollar, Thomas; Laskey, Michael; Leichty, John; Ma, Jie; Nagarajan, Umashankar; Ochiai, Akiyoshi; Petersen, Josh; Shankar, Krishna; Stone, Kevin; Takaoka, Yutaka

doi:10.1109/icra40945.2020.9196677

Cited by 21 publications

(11 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For images, the most common approach is to employ ResNet [171] as the backbone [55,57,84,90,96,97,122,131,172]. In addition to ResNet, various other architectures have been used, including VGG [173] LeNet [174], DenseNet [175] and U-Net [176].…”

Section: Imagesmentioning

confidence: 99%

Deep Learning Approaches to Grasp Synthesis: A Review

Newbury¹,

Gu²,

Lachlan³

et al. 2022

Preprint

View full text Add to dashboard Cite

Grasping is the process of picking an object by applying forces and torques at a set of contacts. Recent advances in deep-learning methods have allowed rapid progress in robotic object grasping. We systematically surveyed the publications over the last decade, with a particular interest in grasping an object using all 6 degrees of freedom of the end-effector pose. Our review found four common methodologies for robotic grasping: sampling-based approaches, direct regression, reinforcement learning, and exemplar approaches. Furthermore, we found two 'supporting methods' around grasping that use deeplearning to support the grasping process, shape approximation, and affordances. We have distilled the publications found in this systematic review (85 papers) into ten key takeaways we consider crucial for future robotic grasping and manipulation research.

show abstract

Section: Imagesmentioning

confidence: 99%

Deep Learning Approaches to Grasp Synthesis: A Review

Newbury¹,

Gu²,

Lachlan³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Humanoid robots that inherit many human body features are seemingly a suitable platform for mimicking human body motions. Application of such anthropomorphic creatures ranges from human interaction [71] to housekeeping teleoperation [3] or even hazardous disaster rescue [56]. With further expansions to smaller-sized humanoid control via motion imitation [35], which highlights the challenges of projecting the human posture to a new morphology.…”

Section: Motion-based Controlmentioning

confidence: 99%

“…The first component of our motion-based control system is a motion retargeting module, which converts the user's motion into the corresponding robot motion. Many prior works have demonstrated successful human-to-humanoid motion mapping [59,66,1,84,71,35,56,31,57,3,10]. However, our problem is unique in the sense that we have to find a mapping function between two very different morphologies without leveraging hand-engineered motion features, such as contact states or centroidal dynamics.…”

Section: Motion Retargetingmentioning

confidence: 99%

Human Motion Control of Quadrupedal Robots using Deep Reinforcement Learning

Kim¹,

Sorokin²,

Lee³

et al. 2022

Preprint

View full text Add to dashboard Cite

A motion-based control interface promises flexible robot operations in dangerous environments by combining user intuitions with the robot's motor capabilities. However, designing a motion interface for non-humanoid robots, such as quadrupeds or hexapods, is not straightforward because different dynamics and control strategies govern their movements. We propose a novel motion control system that allows a human user to operate various motor tasks seamlessly on a quadrupedal robot. We first retarget the captured human motion into the corresponding robot motion with proper semantics using supervised learning and post-processing techniques. Then we apply the motion imitation learning with curriculum learning to develop a control policy that can track the given retargeted reference. We further improve the performance of both motion retargeting and motion imitation by training a set of experts. As we demonstrate, a user can execute various motor tasks using our system, including standing, sitting, tilting, manipulating, walking, and turning, on simulated and real quadrupeds. We also conduct a set of studies to analyze the performance gain induced by each component.(Video 1

show abstract

“…Our approach can be compared to other learning-based methods being used for mobile manipulation. The system developed by [10] applies several learning techniques, combined with a graph-based connection between motion primitives, to complete several tasks. View-invariance of the control policies is encoded in separate object recognition and planning modules.…”

Section: Related Workmentioning

confidence: 99%

Seeing All the Angles: Learning Multiview Manipulation Policies for Contact-Rich Tasks from Demonstrations

Ablett,

Zhai,

Kelly

2021

Preprint

View full text Add to dashboard Cite

Learned visuomotor policies have shown considerable success as an alternative to traditional, hand-crafted frameworks for robotic manipulation tasks. Surprisingly, the extension of these methods to the multiview domain is relatively unexplored. A successful multiview policy could be deployed on a mobile manipulation platform, allowing it to complete a task regardless of its view of the scene. In this work, we demonstrate that a multiview policy can be found through imitation learning by collecting data from a variety of viewpoints. We illustrate the general applicability of the method by learning to complete several challenging multi-stage and contact-rich tasks, from numerous viewpoints, both in a simulated environment and on a real mobile manipulation platform. Furthermore, we analyze our policies to determine the benefits of learning from multiview data compared to learning with data from a fixed perspective. We show that learning from multiview data has little, if any, penalty to performance for a fixed-view task compared to learning with an equivalent amount of fixed-view data. Finally, we examine the visual features learned by the multiview and fixed-view policies. Our results indicate that multiview policies implicitly learn to identify spatially correlated features with a degree of view-invariance.

show abstract

A Mobile Manipulation System for One-Shot Teaching of Complex Tasks in Homes

Cited by 21 publications

References 25 publications

Deep Learning Approaches to Grasp Synthesis: A Review

Deep Learning Approaches to Grasp Synthesis: A Review

Human Motion Control of Quadrupedal Robots using Deep Reinforcement Learning

Seeing All the Angles: Learning Multiview Manipulation Policies for Contact-Rich Tasks from Demonstrations

Contact Info

Product

Resources

About