Unreal mask: one-shot multi-object class-based pose estimation for robotic manipulation using keypoints with a synthetic dataset

Zabihifar, Seyedhassan; Semochkin, Aleksandr; Seliverstova, Elena V.; Efimov, Albert R.

doi:10.1007/s00521-020-05644-6

Cited by 7 publications

(1 citation statement)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The traditional ways for one-shot learning usually fall into two categories: feature extraction and metric learning. For example, Zabihifar's work [15] on multi-object class-based pose estimation is purely relied on synthetic data to get the features that are fed to a nearest-neighbor classifier to recognize the gestures. Shi et al [16] figured out a device-free method to recognize human activity by adopting one-shot learning to the analysis of channel state information of Wi-Fi network.…”

Section: Related Workmentioning

confidence: 99%

TwinNet: A Symmetrically Structured Convolutional Architecture for Identifying Tibetan Characters

Hao

Wang

Lin

et al. 2022

Preprint

View full text Add to dashboard Cite

Although the Tibetan language is widely used, its intelligent application is seriously lagging behind. Most studies on character recognition have almost ignored minority languages like Tibetan. For the purpose of recognizing Tibetan characters, a convolutional architecture named TwinNet is proposed in this work. Specifically, two parallel convolutional sub-networks sharing the same parameters were firstly carefully designed and connected using an energy function, thus achieving Tibetan character recognition via one-shot verification task based on similarity metric. Second, a fuzzy c-means clustering module based on statistical laws of Tibetan characters was integrated into the pipeline of TwinNet, thus greatly reducing the search space at the recognition stage. Third, a Tibetan similar character dataset (TSCD) is constructed after sufficient amount of mining and analysis work, providing data support for training supervised models. The results of binary classification experiment demonstrate that TwinNet can differentiate the similar and dissimilar image pairs with a recall of 0.92, a precision of 0.89, an accuracy of 0.90, an F-1 score of 0.90 and a Kappa coefficient of 0.82. Furthermore, the consistency between the output TwinNet and the subjective evaluation by Tibetan-speaking volunteers is also evaluated. The experimental results support the idea that the similarity evaluation of Tibetan characters by TwinNet is highly consistent with that of humans, the indicator of SROCC, PLCC and RMSE are 0.68, 0.84 and 0.04, respectively. The results of character recognition experiments based on one-shot verification task show that the improved model with the integration of the fuzzy c-means clustering module has an average accuracy of 92% for regular characters and 80% for severely deformed characters.

show abstract

Section: Related Workmentioning

confidence: 99%

TwinNet: A Symmetrically Structured Convolutional Architecture for Identifying Tibetan Characters

Hao

Wang

Lin

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

TRF-Net: a transformer-based RGB-D fusion network for desktop object instance segmentation

Zhang

Shan

et al. 2023

Neural Comput & Applic

View full text Add to dashboard Cite

Multi-Modal Pose Representations for 6-DOF Object Tracking

Majcher,

Kwolek

2024

J Intell Robot Syst

View full text Add to dashboard Cite

Pose estimation methods for robotics should return a distribution of poses rather than just a single pose estimate. Motivated by this, in this work we investigate multi-modal pose representations for reliable 6-DoF object tracking. A neural network architecture for simultaneous object segmentation and estimation of fiducial points of the object on RGB images is proposed. Given a priori probability distribution of object poses a particle filter is employed to estimate the posterior probability distribution of object poses. An advanced observation model relying on matching the projected 3D model with the segmented object and a distance transform-based object representation is used to weight samples representing the probability distribution. Afterwards, the object pose determined by the PnP algorithm is included in the probability distribution via replacing a particle with the smallest weight. Next, a k-means++ algorithm is executed to determine modes in a multi-modal probability distribution. A multi-swarm particle swarm optimization is then executed to determine the finest modes in the probability distribution. A subset of particles for final pose optimization is found in a multi-criteria analysis using the TOPSIS algorithm. They are verified using conflicting criteria that are determined on the basis of object keypoints, segmented object, and the distance transform. On the challenging YCB-Video dataset it outperforms recent algorithms for both object pose estimation and object pose tracking.

show abstract

Unreal mask: one-shot multi-object class-based pose estimation for robotic manipulation using keypoints with a synthetic dataset

Cited by 7 publications

References 28 publications

TwinNet: A Symmetrically Structured Convolutional Architecture for Identifying Tibetan Characters

TwinNet: A Symmetrically Structured Convolutional Architecture for Identifying Tibetan Characters

TRF-Net: a transformer-based RGB-D fusion network for desktop object instance segmentation

Multi-Modal Pose Representations for 6-DOF Object Tracking

Contact Info

Product

Resources

About