Model-Based 3D Contact Geometry Perception for Visual Tactile Sensor

Ji, Jingjing; Liu, Yuting; Ma, Huan

doi:10.3390/s22176470

Cited by 6 publications

(4 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Planar-laser-guided solution with a unique reference point and sensor module mounted with a photoelectric position sensitive detector 16 . Visual-tactile sensor 17 . Optical fibers 18 .…”

Section: Related Studiesmentioning

confidence: 99%

See 1 more Smart Citation

Industrial camera model positioned on an effector for automated tool center point calibration

Konecny,

Beremlijski,

Bailova

et al. 2024

Sci Rep

View full text Add to dashboard Cite

The study presents a novel, full model of an industrial camera suitable for robotic manipulator tool center point (TCP) calibration. The authors propose a new solution which employs a full camera model positioned on the effector of an industrial robotic arm. The proposed full camera model simulates the capture of a calibration pattern for use in automated TCP calibration. The study describes an experimental test robot stand for producing a reference data set, a full camera model, the parameters of a generally known camera obscura model, and a comparison of proposed solution with the camera obscura model. The results are discussed in the context of an innovative approach which features a full camera model to assist the TCP calibration process. The results showed that the full camera model produced greater accuracy, a significant benefit not provided by other state-of-the-art methods. In several cases, the absolute error produced was up to seven times lower than with the state-of-the-art camera obscura model. The error for small rotation (max. of 5$$^\circ $$ ∘ ) and small translation (max. of 20 mm) was 3.65 pixels. The results also highlighted the applicability of the proposed solution in real-life industrial processes.

show abstract

“…Planar-laser-guided solution with a unique reference point and sensor module mounted with a photoelectric position sensitive detector 16 . Visual-tactile sensor 17 . Optical fibers 18 .…”

Section: Related Studiesmentioning

confidence: 99%

“… 15 introduced visual reconstructions using a planar laser, two cameras and a 3D orientation board to verify the accuracy of an active vision system’s measurement fields and calibration. Other methods involve the use of visual-tactile sensors 17 or optical fibers 18 .…”

Section: Related Studiesmentioning

confidence: 99%

Industrial camera model positioned on an effector for automated tool center point calibration

Konecny,

Beremlijski,

Bailova

et al. 2024

Sci Rep

View full text Add to dashboard Cite

show abstract

“…This is mainly because 3D tactile perception units are relatively sparse and can reach a millimeter level but are not sensitive to submillimeter-level perception. Although some optical tactile array sensors can achieve submillimeter sensing ability (Ward-Cherrier et al , 2018; Ji et al , 2022), they are unsuitable for the pressing operation of FPC-t due to the fragile photosensitive element. Due to the lack of temporally continuous information processing, it is still challenging to realize the manipulation and assembly of small objects even when three-axis tactile sensors are used.…”

Section: Related Workmentioning

confidence: 99%

An MS-TCN based spatiotemporal model with three-axis tactile for enhancing flexible printed circuit assembly

Kang,

Cui,

Wang

et al. 2024

RIA

View full text Add to dashboard Cite

Purpose Current flexible printed circuit (FPC) assembly relies heavily on manual labor, limiting capacity and increasing costs. Small FPC size makes automation challenging as terminals can be visually occluded. The purpose of this study is to use 3D tactile sensing to mimic human manual mating skills for enabling sensing offset between FPC terminals (FPC-t) and FPC mating slots (FPC-s) under visual occlusion. Design/methodology/approach The proposed model has three stages: spatial encoding, offset estimation and action strategy. The spatial encoder maps sparse 3D tactile data into a compact 1D feature capturing valid spatial assembly information to enable temporal processing. To compensate for low sensor resolution, consecutive spatial features are input to a multistage temporal convolutional network which estimates alignment offsets. The robot then performs alignment or mating actions based on the estimated offsets. Findings Experiments are conducted on a Redmi Note 4 smartphone assembly platform. Compared to other models, the proposed approach achieves superior offset estimation. Within limited trials, it successfully assembles FPCs under visual occlusion using three-axis tactile sensing. Originality/value A spatial encoder is designed to encode three-axis tactile data into feature maps, overcoming multistage temporal convolution network’s (MS-TCN) inability to directly process such input. Modifying the output to estimate assembly offsets with related motion semantics overcame MS-TCN’s segmentation points output, unable to meet assembly monitoring needs. Training and testing the improved MS-TCN on an FPC data set demonstrated accurate monitoring of the full process. An assembly platform verified performance on automated FPC assembly.

show abstract

“…For instance, Ji et al [24] proposed a novel model-based scheme using a visual-tactile sensor (VTS) [25]. In their study, the sensor compromised a deformable layer that interacted with objects with a depth camera behind the said layer to generate a depth map of the deformation caused by the object.…”

Section: Literature Reviewmentioning

confidence: 99%

Exploring Tactile Temporal Features for Object Pose Estimation during Robotic Manipulation

Galaiya,

Asfour,

Alves de Oliveira

et al. 2023

Sensors

View full text Add to dashboard Cite

Dexterous robotic manipulation tasks depend on estimating the state of in-hand objects, particularly their orientation. Although cameras have been traditionally used to estimate the object’s pose, tactile sensors have recently been studied due to their robustness against occlusions. This paper explores tactile data’s temporal information for estimating the orientation of grasped objects. The data from a compliant tactile sensor were collected using different time-window sample sizes and evaluated using neural networks with long short-term memory (LSTM) layers. Our results suggest that using a window of sensor readings improved angle estimation compared to previous works. The best window size of 40 samples achieved an average of 0.0375 for the mean absolute error (MAE) in radians, 0.0030 for the mean squared error (MSE), 0.9074 for the coefficient of determination (R2), and 0.9094 for the explained variance score (EXP), with no enhancement for larger window sizes. This work illustrates the benefits of temporal information for pose estimation and analyzes the performance behavior with varying window sizes, which can be a basis for future robotic tactile research. Moreover, it can complement underactuated designs and visual pose estimation methods.

show abstract

Model-Based 3D Contact Geometry Perception for Visual Tactile Sensor

Cited by 6 publications

References 26 publications

Industrial camera model positioned on an effector for automated tool center point calibration

Industrial camera model positioned on an effector for automated tool center point calibration

An MS-TCN based spatiotemporal model with three-axis tactile for enhancing flexible printed circuit assembly

Exploring Tactile Temporal Features for Object Pose Estimation during Robotic Manipulation

Contact Info

Product

Resources

About