Elastic Interaction of Particles for Robotic Tactile Simulation

Yikai, Wang; Huang, Wenbing; Fang, Bin; Sun, Fuchun

doi:10.48550/arxiv.2011.11528

Cited by 3 publications

(4 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[16]- [23]), the authors are not aware of any work where a simulation from first principles is utilized to learn tactile feedback control policies. Rather, the mentioned works focus on gathering supervised datasets of tactile images in simulation to train deep neural networks that can predict object position and rotation ( [16], [21]), the force distribution acting on the sensor surface ( [22], [23]), or the three-dimensional mesh of the object in contact ( [17]).…”

Section: A Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Zero-Shot Sim-to-Real Transfer of Tactile Control Policies for Aggressive Swing-Up Manipulation

Sferrazza

D’Andrea

2021

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

This letter aims to show that robots equipped with a vision-based tactile sensor can perform dynamic manipulation tasks without prior knowledge of all the physical attributes of the objects to be manipulated. For this purpose, a robotic system is presented that is able to swing up poles of different masses, radii and lengths, to an angle of 180 • , while relying solely on the feedback provided by the tactile sensor. This is achieved by developing a novel simulator that accurately models the interaction of a pole with the soft sensor. A feedback policy that is conditioned on a sensory observation history, and which has no prior knowledge of the physical features of the pole, is then learned in the aforementioned simulation. When evaluated on the physical system, the policy is able to swing up a wide range of poles that differ significantly in their physical attributes without further adaptation. To the authors' knowledge, this is the first work where a feedback policy from high-dimensional tactile observations is used to control the swing-up manipulation of poles in closed-loop.

show abstract

Section: A Related Workmentioning

confidence: 99%

“…Further, both policies are parametrized using two-layer fully connected neural networks. For the reinforcement learning of the expert policy, the SAC [31] method is employed to find the optimal policy according to (17). The student policy is then deployed to the real-world system without further adaptation.…”

Section: Learning Tactile Control Policiesmentioning

confidence: 99%

Zero-Shot Sim-to-Real Transfer of Tactile Control Policies for Aggressive Swing-Up Manipulation

Sferrazza

D’Andrea

2021

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

show abstract

“…[15]- [22]), the authors are not aware of any work where a simulation from first principles is utilized to learn tactile feedback control policies. Rather, the mentioned works focus on gathering supervised datasets of tactile images in simulation to train deep neural networks that can predict object position and rotation ( [15], [20]), the force distribution acting on the sensor surface ( [21], [22]), or the three-dimensional mesh of the object in contact ( [16]).…”

Section: A Related Workmentioning

confidence: 99%

Zero-shot sim-to-real transfer of tactile control policies for aggressive swing-up manipulation

Sferrazza

D’Andrea

2021

Preprint

View full text Add to dashboard Cite

This paper aims to show that robots equipped with a vision-based tactile sensor can perform dynamic manipulation tasks without prior knowledge of all the physical attributes of the objects to be manipulated. For this purpose, a robotic system is presented that is able to swing up poles of different masses, radii and lengths, to an angle of 180°, while relying solely on the feedback provided by the tactile sensor. This is achieved by developing a novel simulator that accurately models the interaction of a pole with the soft sensor. A feedback policy that is conditioned on a sensory observation history, and which has no prior knowledge of the physical features of the pole, is then learned in the aforementioned simulation. When evaluated on the physical system, the policy is able to swing up a wide range of poles that differ significantly in their physical attributes without further adaptation. To the authors' knowledge, this is the first work where a feedback policy from high-dimensional tactile observations is used to control the swing-up manipulation of poles in closed-loop.

show abstract

“…In order to address the issue of data efficiency, a number of works have focused on generating training data in simulation to extract a model that retains its accuracy when employed in the real world. Examples of such sim-to-real (or sim2real) transfers can be found in the literature for edge prediction [21] and the estimation of the contact pressure [20] and the deformation field [22], [23]. In previous work, a sim-to-real approach was presented to estimate the 3D force distribution [3] for a limited range of scenarios.…”

Section: A Related Workmentioning

confidence: 99%

Sim-to-real for high-resolution optical tactile sensing: From images to 3D contact force distributions

Sferrazza,

D'Andrea

2020

Preprint

View full text Add to dashboard Cite

The images captured by vision-based tactile sensors carry information about high-resolution tactile fields, such as the distribution of the contact forces applied to their soft sensing surface. However, extracting the information encoded in the images is challenging and often addressed with learningbased approaches, which generally require a large amount of training data. This article proposes a strategy to generate tactile images in simulation for a vision-based tactile sensor based on an internal camera that tracks the motion of spherical particles within a soft material. The deformation of the material is simulated in a finite element environment under a diverse set of contact conditions, and spherical particles are projected to a simulated image. Features extracted from the images are mapped to the 3D contact force distribution, with the ground truth also obtained via finite-element simulations, with an artificial neural network that is therefore entirely trained on synthetic data avoiding the need for real-world data collection. The resulting model exhibits high accuracy when evaluated on real-world tactile images, is transferable across multiple tactile sensors without further training, and is suitable for efficient real-time inference.

show abstract

Elastic Interaction of Particles for Robotic Tactile Simulation

Cited by 3 publications

References 18 publications

Zero-Shot Sim-to-Real Transfer of Tactile Control Policies for Aggressive Swing-Up Manipulation

Zero-Shot Sim-to-Real Transfer of Tactile Control Policies for Aggressive Swing-Up Manipulation

Zero-shot sim-to-real transfer of tactile control policies for aggressive swing-up manipulation

Sim-to-real for high-resolution optical tactile sensing: From images to 3D contact force distributions

Contact Info

Product

Resources

About