Active Visuo-Tactile Point Cloud Registration for Accurate Pose Estimation of Objects in an Unknown Workspace

Murali, Prajval Kumar; Gentner, Michael; Kaboli, Mohsen

doi:10.1109/iros51168.2021.9636877

Cited by 11 publications

(10 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To solve the point cloud registration problem for pose estimation, we proposed our linear translation-invariant Quaternion filter (TIQF) in our previous work [7], [8]. Given correspondences, the point cloud registration problem can be defined as:…”

Section: A Translation-invariant Quaternion Filter (Tiqf)mentioning

confidence: 99%

“…However, collection of large amounts of tactile data is time consuming, cumbersome and leads to sensor wear-and-tear. Hence intelligent data collection strategies are required for efficient tactile-based perception [7]- [13].…”

Section: Introductionmentioning

confidence: 99%

“…Similarly, Shannon entropy has been used for selecting actions that provides the maximum discriminatory information for object classification [10], [19]. In our previous works [7], [8] KL-divergence has been used for tactile action selection for object pose estimation.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An Empirical Evaluation of Various Information Gain Criteria for Active Tactile Action Selection for Pose Estimation

Murali

Dahiya

Kaboli

2022

2022 IEEE International Conference on Flexible and Printable Sensors and Systems (FLEPS)

Self Cite

View full text Add to dashboard Cite

Accurate object pose estimation using multi-modal perception such as visual and tactile sensing have been used for autonomous robotic manipulators in literature. Due to variation in density of visual and tactile data, we previously proposed a novel probabilistic Bayesian filter-based approach termed translation-invariant Quaternion filter (TIQF) for pose estimation. As tactile data collection is time consuming, active tactile data collection is preferred by reasoning over multiple potential actions for maximal expected information gain. In this paper, we empirically evaluate various information gain criteria for action selection in the context of object pose estimation. We demonstrate the adaptability and effectiveness of our proposed TIQF pose estimation approach with various information gain criteria. We find similar performance in terms of pose accuracy with sparse measurements across all the selected criteria.

show abstract

Section: A Translation-invariant Quaternion Filter (Tiqf)mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An Empirical Evaluation of Various Information Gain Criteria for Active Tactile Action Selection for Pose Estimation

Murali

Dahiya

Kaboli

2022

2022 IEEE International Conference on Flexible and Printable Sensors and Systems (FLEPS)

Self Cite

View full text Add to dashboard Cite

show abstract

“…The works proposed in [5] refer to this as cross-modal recognition (CMR) or visuo-tactile recognition (VTR), taking inspiration from its psychological definitions [6]. Beyond [5], other examples of a visuo-tactile cross modality can be found in the literature [7], [8]. The work presented in [7] explored a visuo-tactile cross modality to generate tactile images from visual images and vice versa; yet the training of the two systems required both tactile and visual data.…”

Section: Introductionmentioning

confidence: 99%

Visuo-Tactile Recognition of Partial Point Clouds Using PointNet and Curriculum Learning: Enabling Tactile Perception from Visual Data

Parsons

Albini

Martini

et al. 2023

IEEE Robot. Automat. Mag.

View full text Add to dashboard Cite

his article is about recognizing handheld objects from incomplete tactile observations with a classifier trained on only visual representations. Our method is based on the deep learning (DL) architecture PointNet and a curriculum learning (CL) technique for fostering the learning of descriptors robust to partial representations of objects. The learning procedure gradually decomposes the visual point clouds to synthesize sparser and sparser input data for the model. In this manner, we were able to employ one-shot learning, using the decomposed visual point clouds as augmentations, and reduce the data-collection requirement for training. The approach allows for a gradual improvement of prediction accuracy as more tactile data become available.We evaluated the effectiveness of the curriculum strategy on our generated visual and tactile datasets, experimentally showing that the proposed method improved recognition accuracy by up to 23% on partial tactile data and boosted accuracy on full tactile data from 93 to 100%. The curriculum-trained network recognized objects with an accuracy of 80% using only 20% of the tactile data representing the objects, increasing to 100% accuracy on clouds containing at least 60% of the points.

show abstract

“…Furthermore, contrasting to visual perception, tactilebased recognition requires interaction with the objects as data is collected upon contact with the objects [8]. To reduce redundant data collection, temporal costs and human intervention, several approaches have been proposed for performing active data acquisition through information-gain based action selection [10]- [12], [32]- [34]. Leveraging active perception and learning techniques can aid in reducing data collection costs and improve time efficiency for vision-to-tactile crossmodal domain adaptation.…”

mentioning

confidence: 99%

Deep Active Cross-Modal Visuo-Tactile Transfer Learning for Robotic Object Recognition

Murali

Wang

Lee

et al. 2022

IEEE Robot. Autom. Lett.

Self Cite

View full text Add to dashboard Cite

We propose for the first time, a novel deep active visuo-tactile cross-modal full-fledged framework for object recognition by autonomous robotic systems. Our proposed network xAVTNet is actively trained with labelled point clouds from a vision sensor with one robot and tested with an active tactile perception strategy to recognise objects never touched before using another robot. We propose a novel visuo-tactile loss (VTLoss) to minimise the discrepancy between the visual and tactile domains for unsupervised domain adaptation. Our framework leverages the strengths of deep neural networks for cross-modal recognition along with active perception and active learning strategies for increased efficiency by minimising redundant data collection. Our method is extensively evaluated on a real robotic system and compared against baselines and other state-of-art approaches. We demonstrate clear outperformance in recognition accuracy compared to the state-of-art visuo-tactile cross-modal recognition method.

show abstract

Active Visuo-Tactile Point Cloud Registration for Accurate Pose Estimation of Objects in an Unknown Workspace

Cited by 11 publications

References 28 publications

An Empirical Evaluation of Various Information Gain Criteria for Active Tactile Action Selection for Pose Estimation

An Empirical Evaluation of Various Information Gain Criteria for Active Tactile Action Selection for Pose Estimation

Visuo-Tactile Recognition of Partial Point Clouds Using PointNet and Curriculum Learning: Enabling Tactile Perception from Visual Data

Deep Active Cross-Modal Visuo-Tactile Transfer Learning for Robotic Object Recognition

Contact Info

Product

Resources

About