Estimating the deformability of elastic materials using optical flow and position-based dynamics

Abstract

Abstract-Knowledge of the physical properties of objects is essential in a wide range of robotic manipulation scenarios. A robot may not always be aware of such properties prior to interaction. If an object is incorrectly assumed to be rigid, it may exhibit unpredictable behavior when grasped. In this paper, we use vision based observation of the behavior of an object a robot is interacting with and use it as the basis for estimation of its elastic deformability. This is estimated in a local region around the interaction point using a physics simulator. We use optical flow to estimate the parameters of a position-based dynamics simulation using meshless shape matching (MSM). MSM has been widely used in computer graphics due to its computational efficiency, which is also important for closed-loop control in robotics. In a controlled experiment we demonstrate that our method can qualitatively estimate the physical properties of objects with different degrees of deformability.

Multi-Modal Sensing and Robotic Manipulation of Non-Rigid Objects: A Survey

“…Güler et al [5] used a Logitech HD Pro C920 Webcam (Logitech, Newark, CA, USA) to develop 2D deformation tracking under a robotic pushing scenario. The system used a geometric scheme that simulated the physical object behavior based on a position-based dynamics (PBD) model.…”

“…Fugl et al [9] Frank et al [25] Leizea et al [10] Petit et al [26] Lin et al [11] Güler et al [5] Caccamo et al [29] Hui et al [6] Cretu et al [22] Arriola-Rios and Wyatt [21] Drimus et al [14] Mira et al [16] Delgado et al [18] Navarro-Alarcon et al [12] Alambeigi et al [13]…”

“…In order to improve the stability issues of physics-based models, Güler et al 's [5] system used a position-based dynamics model (PBD) to simulate the deformation. The model is a geometric simulation method called meshless shape matching, originally presented in [37].…”

“…A learning phase based on Kernel Density Estimation (KDE) method is also considered, which involved the identification of the most suitable manipulation actions and the current state of the object accordingly. Güler et al [47] updated their previous approach [5] presented in Section 3.1.2 to include a FEM model that generated ground truth data of various deformation parameters. The robustness of the system was improved by adding quadratic deformation (i.e., twist and bend).…”

Active perception and modeling of deformable surfaces using Gaussian processes and position-based dynamics

“…1. It is showed in [20] that by matching real-world observation and MSM simulation, the deformability of objects can be estimated in a controlled 2D experimental setup. Here instead, we map the deformability of a surface with heterogeneous material properties by minimizing the error between the model prediction and observed deformations in 3D space.…”