Real-to-Sim Registration of Deformable Soft Tissue with Position-Based Dynamics for Surgical Robot Autonomy

Liu, Fei; Han, Yunhai; Lu, Jingpei; Richter, Florian; Yip, Michael C.

doi:10.1109/icra48506.2021.9561177

Cited by 30 publications

(19 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While PBD's original development was towards complex rendering scenes for particle simulation, fluid, and soft bodies, rigid body simulation has been detailed through the formulation of position-based dynamics [16], more recently, applications in robotics has been investigated. This includes deformable tissue modeling [17], [18], and blood fluid simulation [19], [20]. The obvious advantage of PBDs towards robot learning is that their speed, robustness, and native capability for defining constraints make it a scalable tool for eventually tackling complex scenarios and environments.…”

Section: A Optimization In Roboticsmentioning

confidence: 99%

“…The goal is to determine the control inputs to the simulated dynamics model such that the error between a measured trajectory and a modeled trajectory are minimized. This is the real-to-sim problem that has been of more recent interest as a method for imitation and transfer learning [17], [53]. MPC is used to determine the control inputs iteratively since solving the full trajectory worth of control inputs at once that results in matching trajectories between sim and real would be difficult.…”

Section: Real-to-sim Mpc Torque Estimationmentioning

confidence: 99%

See 1 more Smart Citation

Parameter Identification and Motion Control for Articulated Rigid Body Robots Using Differentiable Position-based Dynamics

Liu¹,

Li²,

Lu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Section: A Optimization In Roboticsmentioning

confidence: 99%

Section: Real-to-sim Mpc Torque Estimationmentioning

confidence: 99%

Parameter Identification and Motion Control for Articulated Rigid Body Robots Using Differentiable Position-based Dynamics

Liu¹,

Li²,

Lu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

“…We consider the real-to-sim problem of inferring parameters of general-purpose simulators from real observations such that the gap between reality and simulation is reduced [1]- [4]. A common approach to solving this problem is to train an inverse model on data generated with a blackbox (non-differentiable) simulator using a wide range of parameter settings.…”

Section: Introductionmentioning

confidence: 99%

DiffCloud: Real-to-Sim from Point Clouds with Differentiable Simulation and Rendering of Deformable Objects

Sundaresan¹,

Antonova²,

Bohg³

2022

Preprint

View full text Add to dashboard Cite

Research in manipulation of deformable objects is typically conducted on a limited range of scenarios, because handling each scenario on hardware takes significant effort. Realistic simulators with support for various types of deformations and interactions have the potential to speed up experimentation with novel tasks and algorithms. However, for highly deformable objects it is challenging to align the output of a simulator with the behavior of real objects. Manual tuning is not intuitive, hence automated methods are needed. We view this alignment problem as a joint perception-inference challenge and demonstrate how to use recent neural network architectures to successfully perform simulation parameter inference from real point clouds. We analyze the performance of various architectures, comparing their data and training requirements. Furthermore, we propose to leverage differentiable point cloud sampling and differentiable simulation to significantly reduce the time to achieve the alignment. We employ an efficient way to propagate gradients from point clouds to simulated meshes and further through to the physical simulation parameters, such as mass and stiffness. Experiments with highly deformable objects show that our method can achieve comparable or better alignment with real object behavior, while reducing the time needed to achieve this by more than an order of magnitude. Videos and supplementary material are available at https://tinyurl.com/diffcloud.

show abstract

“…For problems in grasping soft objects, modeling and simulation of deformable objects have recently been considerable interest in many robotic applications [2], [3], to manipulating soft tissue [4], [5], cloth [6], [7], and even fluids [8], [9]. Among soft structures, deformable linear objects (DLOs), including rope-like objects, strings, cables, beams, etc., are studied (cable routing [10], wire insertion [11], flexible rope [12], and knotting of surgical thread [13]).…”

Section: Introductionmentioning

confidence: 99%

“…Next, the authors added scale parameters for rope-like objects to preserve volume consistency [45]. Although several works have been applied for real-tosim tasks of soft deformable tissue [4], [5], rigid articulated robots [46], and fluids [8], [47], with constraint-based formulation of PBD or XPBD. However, there is a missing combination of modeling and control approaches applied to rope-like objects in literature, exploiting the capability of XPBD.…”

Section: Introductionmentioning

confidence: 99%

Differentiable Robotic Manipulation of Deformable Rope-like Objects Using Compliant Position-based Dynamics

Liu¹,

Entong²,

Lu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Robot manipulation of rope-like objects is an interesting problem that has some critical applications, such as autonomous robotic suturing. Solving for and controlling rope is difficult due to the complexity of rope physics and the challenge of building fast and accurate models of deformable materials. While more data-driven approaches have become more popular for finding controllers that learn to do a single task, there is still a strong motivation for a modelbased method that could be used to solve a large variety of optimization problems. Towards this end, we introduced compliant, position-based dynamics (XPBD) to model rope-like objects. Using geometric constraints, the model can represent the coupling of shear/stretch and bend/twist effects. Of crucial importance is that our formulation is differentiable, which can solve parameter estimation problems and improve the matching of rope physics to real-life scenarios (i.e., the realto-sim problem). For the generality of rope-like objects, two different solvers are proposed to handle the inextensible and extensible effects of varied material stiffness for the rope. We demonstrate our framework's robustness and accuracy on realto-sim experimental setups using the Baxter robot and the da Vinci research kit (DVRK) [1]. Our work leads to a new path for robotic manipulation of the deformable rope-like object taking advantage of the ready-to-use gradients.

show abstract

Real-to-Sim Registration of Deformable Soft Tissue with Position-Based Dynamics for Surgical Robot Autonomy

Cited by 30 publications

References 22 publications

Parameter Identification and Motion Control for Articulated Rigid Body Robots Using Differentiable Position-based Dynamics

Parameter Identification and Motion Control for Articulated Rigid Body Robots Using Differentiable Position-based Dynamics

DiffCloud: Real-to-Sim from Point Clouds with Differentiable Simulation and Rendering of Deformable Objects

Differentiable Robotic Manipulation of Deformable Rope-like Objects Using Compliant Position-based Dynamics

Contact Info

Product

Resources

About