Predicting the Force Map of an ERT-Based Tactile Sensor Using Simulation and Deep Networks

Abstract

Electrical resistance tomography (ERT) can be used to create large-scale soft tactile sensors that are flexible and robust. Good performance requires a fast and accurate mapping from the sensor's sequential voltage measurements to the distribution of force across its surface. However, particularly with multiple contacts, this task is challenging for both previously developed approaches: physics-based modeling and end-to-end data-driven learning. Some promising results were recently achieved using sim-to-real transfer learning, but estimating multiple contact locations and accurate contact forces remains difficult because simulations tend to be less accurate with a high number of contact locations and/or high force. This paper introduces a modular hybrid method that combines simulation data synthesized from an electromechanical finite element model with real measurements collected from a new ERT-based tactile sensor. We use about 290 000 simulated and 90 000 real measurements to train two deep neural networks: the first (Transfer-Net) captures the inevitable gap between simulation and reality, and the second (Recon-Net) reconstructs contact forces from voltage measurements. The number of contacts, contact locations, force magnitudes, and contact diameters are evaluated for a manually collected multi-contact dataset of 150 measurements. Our modular pipeline's results outperform predictions by both a physics-based model and end-to-end learning. Note to Practitioners-ERT-based tactile sensors use highspeed voltage measurements from electrodes distributed over a piezoresistive area to output a force map that shows where contact is occurring, and how strong each contact is. Such sensors Manuscript

E-Skin Using Fringing Field Electrical Impedance Tomography with an Ionic Liquid Domain

“…In Figure 9C, by combining a location detection via the EIT image and the calibration setting in Figure 9B, we can produce a single calibration form as shown in Figure 9C. A data driven approach to force quantification is shown in [14]. Figure 9 shows an approach that combines the position of touch with the calibration leading to a single calibrating plot.…”

“…The alternative to such artificial skins, is to use an array of discrete sensors, however, the new EIT-based technology offers the following advantages: minimal wiring; flexible; scalable; continuous sensing across the domain; easy to manufacture; low power consumption and low cost [3,12]. The e-skin has become a very important area of development in the past few years [13][14][15][16][17][18], whether it is for robots that are like humans, or robots that needs to work in a safe environment with humans or other robots. This paper covers an important area of soft robotics by providing methods of performance evaluation.…”

A Hybrid FPGA-GPU-based Hardware Accelerator for High Throughput 2D Electrical Impedance Tomography

“…Similar to [7], the processing was done using a general-purpose IBM-compatible PC, while the data was acquired using an Avnet Zedboard with Zynq 7000 (SOC module) at a rate of up to 30 fps. Researchers at the Max Planck Institute for Intelligent Systems designed a 28-electrode EIT system with a current excitation of 24 kHz to yield a 30 fps throughput for predicting the force map of an ERT-based tactile sensor [9]. Another similar system based on Xilinx's FPGA technology was suggested in [10] to yield a throughput of up to 120 fps.…”

A novel neural network-based data acquisition system targeting high-speed electrical impedance tomography systems

“…A 10 kHz excitation signal with a 100 kHz sampling frequency was used, and 4 cycles per channel were considered for a single channel to increase the SNR, which led to a maximal throughput of only 11 fps. Authors in [3], designed a 28-electrode ERT system with a current excitation of 24 kHz to yield a 30 fps throughput for predicting the force map of an ERT-based tactile sensor. This may not be enough to provide an accurate map of the stresses applied to large areas, which is very common.…”