Efficient Recognition and 6D Pose Tracking of Markerless Objects with RGB-D and Motion Sensors on Mobile Devices

Huang, Sheng Chu; Huang, Wei; Lu, Yi; Tsai, Ming Han; Lin, I-Chen; Lau, Yo Chung; Liu, Hsu Hang

doi:10.5220/0007692603750382

Cited by 4 publications

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…6D Poses are an explicit means of localization describing an object's six-degree-of-freedom (6D) pose in space. 6D pose estimation is used in a number of real-world applications, such as robotic grasping [15] and augmented reality [16]. Due to the nature of dimensionality reduction during the image generation process (i.e.…”

Section: Related Workmentioning

confidence: 99%

Multi-Class Object Detection Using 2D Poses

Mayershofer

Hammami

Fottner

2020

2020 19th IEEE International Conference on Machine Learning and Applications (ICMLA)

View full text Add to dashboard Cite

Object detection (OD) methods are finding application in various fields. The OD problem can be divided into two sub-problems, namely object classification and localization. While the former aims to answer the question what class a given object belongs to, the latter focuses on locating an object within a given image. For localization, both implicit representations, which border the object and its features (e.g. bounding boxes, polygons and masks), and explicit representations, which describe the object's pose in an image (e.g. 6D pose, keypoints), are used. The 2D pose is a simple, yet effective representation that has so far been overlooked. In this paper, we therefore motivate and formulate the use of 2D poses for object localization. Furthermore, we present RetinaNet-2DP, an anchor-based convolutional neural network (CNN) that is capable of detecting objects using 2D poses. To do so, we propose the idea of Anchor Poses and the Gaussian Kernel Distance as a similarity metric between poses. Experiments on the DOTA dataset and two robotics use cases from industry emphasize the performance of the network architecture and more generally demonstrate the potential of the proposed localization representation. Finally, we critically assess our findings and present an outlook of future work.

show abstract

Section: Related Workmentioning

confidence: 99%

Multi-Class Object Detection Using 2D Poses

Mayershofer

Hammami

Fottner

2020

2020 19th IEEE International Conference on Machine Learning and Applications (ICMLA)

View full text Add to dashboard Cite

show abstract

Real-Time Object Pose Tracking System With Low Computational Cost for Mobile Devices

Lau,

Tseng,

Kao

et al. 2023

J. Ind. Sea. Pos. Nav.

Self Cite

View full text Add to dashboard Cite

Real-time object pose estimation and tracking is challenging but essential for some emerging applications such as augmented reality (AR). In general, state-of-the-art methods address this problem using deep neural networks which indeed yield satisfactory results. Nevertheless, the high computational cost of these methods makes them unsuitable for mobile devices where real-world applications usually take place. We propose real-time object pose tracking system with low computational cost for mobile devices. It is a monocular inertial-assisted-visual system with a client-server architecture connected by high-speed networking. Inertial measurement unit (IMU) pose propagation is performed on the client side for fast pose tracking, and RGB image-based 3D object pose estimation is performed on the server side to obtain accurate poses, after which the pose is sent to the client side for refinement, where we propose a bias self-correction mechanism to reduce the drift. We also propose a fast and effective pose inspection algorithm to detect tracking failures and incorrect pose estimation. In this way, the pose updates rapidly even within 5 ms on low-level devices, making it possible to support realtime tracking for applications. Additionally, an object pose dataset with RGB images and IMU measurements is delivered for evaluation. Experiments also show our method performs well with both accuracy and robustness.

show abstract

Evaluation of the use of box size priors for 6D plane segment tracking from point clouds with applications in cargo packing

Camacho-Muñoz,

Nope Rodríguez,

Loaiza-Correa

et al. 2024

J Image Video Proc.

View full text Add to dashboard Cite

This paper addresses the problem of 6D pose tracking of plane segments from point clouds acquired from a mobile camera. This is motivated by manual packing operations, where an opportunity exists to enhance performance, aiding operators with instructions based on augmented reality. The approach uses as input point clouds, by its advantages for extracting geometric information relevant to estimating the 6D pose of rigid objects. The proposed algorithm begins with a RANSAC fitting stage on the raw point cloud. It then implements strategies to compute the 2D size and 6D pose of plane segments from geometric analysis of the fitted point cloud. Redundant detections are combined using a new quality factor that predicts point cloud mapping density and allows the selection of the most accurate detection. The algorithm is designed for dynamic scenes, employing a novel particle concept in the point cloud space to track detections' validity over time. A variant of the algorithm uses box size priors (available in most packing operations) to filter out irrelevant detections. The impact of this prior knowledge is evaluated through an experimental design that compares the performance of a plane segment tracking system, considering variations in the tracking algorithm and camera speed (onboard the packing operator). The tracking algorithm varies at two levels: algorithm ( A wpk ), which integrates prior knowledge of box sizes, and algorithm ( A woutpk ), which assumes ignorance of box properties. Camera speed is evaluated at low and high speeds. Results indicate increments in the precision and F1-score associated with using the A wpk algorithm and consistent performance across both velocities. These results confirm the enhancement of the performance of a tracking system in a real-life and complex scenario by including previous knowledge of the elements in the scene. The proposed algorithm is limited to tracking plane segments of boxes fully supported on surfaces parallel to the ground plane and not stacked. Future works are proposed to include strategies to resolve this limitation.

show abstract

Efficient Recognition and 6D Pose Tracking of Markerless Objects with RGB-D and Motion Sensors on Mobile Devices

Cited by 4 publications

References 0 publications

Multi-Class Object Detection Using 2D Poses

Multi-Class Object Detection Using 2D Poses

Real-Time Object Pose Tracking System With Low Computational Cost for Mobile Devices

Evaluation of the use of box size priors for 6D plane segment tracking from point clouds with applications in cargo packing

Contact Info

Product

Resources

About