Visual Interaction Force Estimation Based on Time-Sensitive Dual-Resolution Learning Network

Abstract

Haptic force feedback is an important perception method for humans to understand the surrounding environment. It can estimate tactile force in real time and provide appropriate feedback. It has important research value for robot-assisted minimally invasive surgery, interactive tactile robots, and other application fields. However, most of the existing noncontact visual power estimation methods are implemented using traditional machine learning or 2D/3D CNN combined with LSTM. Such methods are difficult to fully extract the contextual spatiotemporal interaction semantic information of consecutive multiple frames of images, and their performance is limited. To this end, this paper proposes a time-sensitive dual-resolution learning network-based force estimation model to achieve accurate noncontact visual force prediction. First, we perform continuous frame normalization processing on the robot running the video captured by the camera and use the hybrid data augmentation to improve the data diversity; secondly, a deep semantic interaction model is constructed based on the time-sensitive dual-resolution learning network, which is used to automatically extract the deep spatiotemporal semantic interaction information of continuous multiframe images; finally, we construct a simplified prediction model to realize the efficient estimation of interaction force. The results based on the large-scale robot hand interaction dataset show that our method can estimate the interaction force of the robot hand more accurately and faster. The average prediction MSE reaches 0.0009 N, R 2 reaches 0.9833, and the average inference time for a single image is 6.5532 ms; in addition, our method has good prediction generalization performance under different environments and parameter settings.

Interaction Motion Control on Tri-finger Pneumatic Grasper using Variable Convergence Rate Prescribed Performance Impedance Control with Pressure-based Force Estimator

“…12 with accordance to the interaction between both joint angular of the finger and pressure within the pneumatic piston with compensator 𝐺 𝑇 . Recognized as a major element across investigated estimation methods with artifical intelligence in [56][57][58], the processing interval has been optimized in response to the exclusion of iteration process from its algorithmic settings. Sole mechanistic implementation of the structure's geometry and Jacobian for the currently examined method has also guaranteed reduced modelling complexity against the comparable works of [11,12,59,60] where actual sensor reading was selectively applied for results verification.…”