Interleaving Planning and Control for Deformable Object Manipulation

McConachie, Dale; Ruan, Mengyao; Berenson, Dmitry

doi:10.1007/978-3-030-28619-4_68

Cited by 13 publications

(16 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A preliminary version of this work was presented in McConachie et al (2017). This article extends this work by adding an additional experiment on a physical robotic system as well as a proof of the probabilistic completeness of our planning method.…”

Section: Introductionmentioning

confidence: 80%

Manipulating deformable objects by interleaving prediction, planning, and control

McConachie

Dobson

Ruan

et al. 2020

The International Journal of Robotics Research

Self Cite

View full text Add to dashboard Cite

We present a framework for deformable object manipulation that interleaves planning and control, enabling complex manipulation tasks without relying on high-fidelity modeling or simulation. The key question we address is when should we use planning and when should we use control to achieve the task? Planners are designed to find paths through complex configuration spaces, but for highly underactuated systems, such as deformable objects, achieving a specific configuration is very difficult even with high-fidelity models. Conversely, controllers can be designed to achieve specific configurations, but they can be trapped in undesirable local minima owing to obstacles. Our approach consists of three components: (1) a global motion planner to generate gross motion of the deformable object; (2) a local controller for refinement of the configuration of the deformable object; and (3) a novel deadlock prediction algorithm to determine when to use planning versus control. By separating planning from control we are able to use different representations of the deformable object, reducing overall complexity and enabling efficient computation of motion. We provide a detailed proof of probabilistic completeness for our planner, which is valid despite the fact that our system is underactuated and we do not have a steering function. We then demonstrate that our framework is able to successfully perform several manipulation tasks with rope and cloth in simulation, which cannot be performed using either our controller or planner alone. These experiments suggest that our planner can generate paths efficiently, taking under a second on average to find a feasible path in three out of four scenarios. We also show that our framework is effective on a 16-degree-of-freedom physical robot, where reachability and dual-arm constraints make the planning more difficult.

show abstract

Section: Introductionmentioning

confidence: 80%

Manipulating deformable objects by interleaving prediction, planning, and control

McConachie

Dobson

Ruan

et al. 2020

The International Journal of Robotics Research

Self Cite

View full text Add to dashboard Cite

show abstract

“…One approach to using local controllers is model-based servoing [48,54], where the end-effector is controlled to a goal location instead of explicit planning. However, since the controller is optimized over simple dynamics, it often gets stuck in local minima with more complex dynamics [32]. To solve this, several works [3,31] have proposed Jacobian controllers that do not need explicit models, while [17,16] have proposed learning-based techniques for servoing.…”

Section: Related Work a Deformable Object Manipulationmentioning

confidence: 99%

Learning Predictive Representations for Deformable Objects Using Contrastive Estimation

Yan,

Vangipuram,

Abbeel

et al. 2020

Preprint

View full text Add to dashboard Cite

Fig. 1: Example trajectories using our contrastive forward model for rope and cloth manipulation. The top two rows show rope manipulation from different start states to different goal states, while the bottom two rows show cloth manipulation using different colored cloths. Note that in the last row, the robot is manipulating a white cloth, but are method is able to still use a blue cloth as the goal image.

show abstract

“…The deformable object handling problem has been studied via classical methods such as motion planning and manipulation [ 23 ]. There has been recent interest in combining deep generative models with structured dynamical systems in the context of variational autoencoders, where the latent space is continuous [ 24 ]. Watter et al [ 25 ] used such models to perform the planning via learning latent linear dynamics and exploiting a linear quadratic Gaussian control algorithm.…”

Section: Related Workmentioning

confidence: 99%

Action Generative Networks Planning for Deformable Object with Raw Observations

Sheng

Jin

et al. 2021

Sensors

View full text Add to dashboard Cite

Synthesizing plans for a deformable object to transit from initial observations to goal observations, both of which are represented by high-dimensional data (namely “raw” data), is challenging due to the difficulty of learning abstract state representations of raw data and transition models of continuous states and continuous actions. Even though there have been some approaches making remarkable progress regarding the planning problem, they often neglect actions between observations and are unable to generate action sequences from initial observations to goal observations. In this paper, we propose a novel algorithm framework, namely AGN. We first learn a state-abstractor model to abstract states from raw observations, a state-generator model to generate raw observations from states, a heuristic model to predict actions to be executed in current states, and a transition model to transform current states to next states after executing specific actions. After that, we directly generate plans for a deformable object by performing the four models. We evaluate our approach in continuous domains and show that our approach is effective with comparison to state-of-the-art algorithms.

show abstract

Interleaving Planning and Control for Deformable Object Manipulation

Cited by 13 publications

References 22 publications

Manipulating deformable objects by interleaving prediction, planning, and control

Manipulating deformable objects by interleaving prediction, planning, and control

Learning Predictive Representations for Deformable Objects Using Contrastive Estimation

Action Generative Networks Planning for Deformable Object with Raw Observations

Contact Info

Product

Resources

About