Vision-based interaction force estimation for robot grip motion without tactile/force sensor

Ko, Dae-Kwan; Lee, Kang-Won; Lee, Dong Han; Lim, Soo-Chul

doi:10.1016/j.eswa.2022.118441

Cited by 15 publications

(4 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The first step of encoder preprocessing is to calculate the body speed observation value to determine the encoder error term in the joint initialization and optimization of vision Inertial measurement unit encoder, and determine the degree of wheel slip through the calculation of robot slip factor. The linear and angular velocities of the robot body in the encoder coordinates are calculated as shown in equation ( 15) [18]. ω , respectively; The distance between two wheels is represented by D .…”

Section: B Improved Algorithm In View Of Multi-sensor Fusionmentioning

confidence: 99%

Improved ORB-SLAM2 Mobile Robot Vision Algorithm Based on Multiple Feature Fusion

Hu,

Zhu,

Wang

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Traditional wheeled robot vision algorithms suffer from low texture tracking failures. Therefore, this study proposes a vision improvement algorithm for mobile robots in view of multi feature fusion; This algorithm introduces line surface features and Manhattan Frame on the basis of traditional algorithms, and proposes an improved algorithm in view of multi-sensor fusion to improve tracking accuracy. The experiment shows that the average Root-mean-square deviation of the position of the improved mobile robot vision algorithm in view of multi feature fusion is 0.02 in nine data packets of the Tum dataset; The average Root-mean-square deviation of the position of the data packet successfully tracked by the traditional wheeled robot vision algorithm is 0.016; It improved the average accuracy by 11.11%, which is 31.03% higher than the average accuracy of the Manhattan wheeled robot vision algorithm. Compared to the multi feature fusion based vision improvement algorithm for mobile robots and the closed-loop detection based multi-sensor improvement algorithm, the accuracy of the closed-loop detection based multi-sensor improvement algorithm has increased by 0.655% and 10.47%, respectively. The outcomes indicate that the improved algorithm can improve the accuracy of mobile robot tracking, thereby expanding its application range. INDEX TERMSMulti feature fusion, Mobile robots, Visual algorithms, Multi sensor fusion, Encoder

show abstract

Section: B Improved Algorithm In View Of Multi-sensor Fusionmentioning

confidence: 99%

Improved ORB-SLAM2 Mobile Robot Vision Algorithm Based on Multiple Feature Fusion

Hu,

Zhu,

Wang

et al. 2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Ko et al developed a vision-based system to estimate the interaction forces between the robot grip and objects by combining RGB-D images, robot positions, and motor currents. By incorporating proprioceptive feedback with visual data, the proposed model achieves high accuracy in force estimation [17]. In the context of smart manufacturing, Chen et al develop a real-time milling force monitoring system, using sensory data to accurately estimate the forces involved in the process, thus enabling real-time adjustment to optimize the cutting operation [18].…”

Section: Introductionmentioning

confidence: 99%

AI-Driven Virtual Sensors for Real-Time Dynamic Analysis of Mechanisms: A Feasibility Study

Fabiocchi,

Giulietti,

Carnevale

et al. 2024

Machines

View full text Add to dashboard Cite

The measurement of the ground forces on a real structure or mechanism in operation can be time-consuming and expensive, particularly when production cannot be halted to install sensors. In cases in which disassembling the parts of the system to accommodate sensor installation is neither feasible nor desirable, observing the structure or mechanism in operation and quickly deducing its force trends would facilitate monitoring activities in industrial processes. This opportunity is gradually becoming a reality thanks to the coupling of artificial intelligence (AI) with design techniques such as the finite element and multi-body methods. Properly trained inferential models could make it possible to study the dynamic behavior of real systems and mechanisms in operation simply by observing them in real time through a camera, and they could become valuable tools for investigation during the operation of machinery and devices without the use of additional sensors, which are difficult to use and install. In this paper, the idea presented is developed and applied to a simple mechanism for which the reaction forces during operating conditions are to be determined. This paper explores the implementation of an innovative vision-based virtual sensor that, through data-driven training, is able to emulate traditional sensing solutions for the estimation of reaction forces. The virtual sensor and relative inferential model is validated in a scenario as close to the real world as possible, taking into account interfering inputs that add to the measurement uncertainty, as in a real-world measurement scenario. The results indicate that the proposed model has great robustness and accuracy, as evidenced by the low RMSE values in predicting the reaction forces. This demonstrates the model’s effectiveness in reproducing real-world scenarios, highlighting its potential in the real-time estimation of ground reaction forces in industrial settings. The success of this vision-based virtual sensor model opens new avenues for more robust, accurate, and cost-effective solutions for force estimation, addressing the challenges of uncertainty and the limitations of physical sensor deployment.

show abstract

“…Lee et al [ 39 ] conducted a study in which they used data collected from an accelerometer and gyroscope sensors built into a smartphone as input values for a neural network to recognize the touched area on a smartphone, instead of using the tactile signal from the fingertip. Lee et al [ 40 ] and Ko et al [ 41 ] proposed methods that estimate the interaction force between a robot and an object while the robot grasps the object using visual images, robot position, and electrical current, without relying on tactile signals. Proprioception is necessary for tactile perception in humans.…”

Section: Introductionmentioning

confidence: 99%

Effects of Sensing Tactile Arrays, Shear Force, and Proprioception of Robot on Texture Recognition

Yang

Kim

Lim

2023

Sensors

Self Cite

View full text Add to dashboard Cite

In robotics, tactile perception is important for fine control using robot grippers and hands. To effectively incorporate tactile perception in robots, it is essential to understand how humans use mechanoreceptors and proprioceptors to perceive texture. Thus, our study aimed to investigate the impact of tactile sensor arrays, shear force, and the positional information of the robot’s end effector on its ability to recognize texture. A deep learning network was employed to classify tactile data from 24 different textures that were explored by a robot. The input values of the deep learning network were modified based on variations in the number of channels of the tactile signal, the arrangement of the tactile sensor, the presence or absence of shear force, and the positional information of the robot. By comparing the accuracy of texture recognition, our analysis revealed that tactile sensor arrays more accurately recognized the texture compared to a single tactile sensor. The utilization of shear force and positional information of the robot resulted in an improved accuracy of texture recognition when using a single tactile sensor. Furthermore, an equal number of sensors placed in a vertical arrangement led to a more accurate distinction of textures during exploration when compared to sensors placed in a horizontal arrangement. The results of this study indicate that the implementation of a tactile sensor array should be prioritized over a single sensor for enhanced accuracy in tactile sensing, and the use of integrated data should be considered for single tactile sensing.

show abstract

Vision-based interaction force estimation for robot grip motion without tactile/force sensor

Cited by 15 publications

References 27 publications

Improved ORB-SLAM2 Mobile Robot Vision Algorithm Based on Multiple Feature Fusion

Improved ORB-SLAM2 Mobile Robot Vision Algorithm Based on Multiple Feature Fusion

AI-Driven Virtual Sensors for Real-Time Dynamic Analysis of Mechanisms: A Feasibility Study

Effects of Sensing Tactile Arrays, Shear Force, and Proprioception of Robot on Texture Recognition

Contact Info

Product

Resources

About