A modular framework for model-based visual tracking using edge, texture and depth features

Trinh, Souriya; Spindler, Fabien; Marchand, Éric; Chaumette, François

doi:10.1109/iros.2018.8594003

Cited by 25 publications

(12 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The rigid registration is a joint minimization of two error terms: depth based geometric error and keypoint based feature tracking [22]. The two error terms are explained below:…”

Section: Rigid Registrationmentioning

confidence: 99%

Tracking of Non-Rigid Objects using RGB-D Camera

Sengupta¹,

Krupa²,

Marchand³

2019

2019 IEEE International Conference on Systems, Man and Cybernetics (SMC)

Self Cite

View full text Add to dashboard Cite

A method to accurately track deformable objects using a RGB-D camera with the help of a coarse object model is presented in this paper. The deformation model is based on corotational FEM formulation. The physical model of the object does not need to be exact, nor do we require the precise physical properties for accurately tracking the object. The position of the vertices of the surface mesh of the tracked object is deformed using a set of virtual forces. A point-to-plane distance based geometric error between the pointcloud and the mesh is minimized with respect to these virtual forces. The point of application of force is determined by analysis of the error obtained from rigid tracking, which is done in parallel with the non-rigid tracking. This architecture also enables the overall system to be realtime. The proposed approach is evaluated on a synthetic data with ground-truth for deformation at every frame, as well as on real data.

show abstract

“…The rigid registration is a joint minimization of two error terms: depth based geometric error and keypoint based feature tracking [22]. The two error terms are explained below:…”

Section: Rigid Registrationmentioning

confidence: 99%

Tracking of Non-Rigid Objects using RGB-D Camera

Sengupta¹,

Krupa²,

Marchand³

2019

2019 IEEE International Conference on Systems, Man and Cybernetics (SMC)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Only a few model-free tracking and reconstruction algorithm are publicly available along with their open source code (e.g: [16], [18]), but none of them are directly applicable to model-based object tracking with RGB-D data. We compare the proposed method with two recent approaches: a) 'edge + keypoint + depth tracker'from [25] (denoted in the figures as ViSP), and b) stacked error minimization of point-to-plane distance with photometric error, without using keyframes (denoted as No KF). No KF is close to the tracking module of [18].…”

Section: Resultsmentioning

confidence: 99%

RGB-D Tracking of Complex Shapes Using Coarse Object Models

Sengupta¹,

Krupa²,

Marchand³

2019

2019 IEEE International Conference on Image Processing (ICIP)

Self Cite

View full text Add to dashboard Cite

This paper presents a framework for accurately tracking objects of complex shapes with joint minimization of geometric and photometric parameters using a coarse 3D object model with the RGB-D cameras. Tracking with coarse 3D model is remarkably useful for industrial applications. A technique is proposed that uses a combination of point-to-plane distance minimization and photometric error minimization to track objects accurately. The concept of 'keyframes' are used in this system of object tracking for minimizing drift. The proposed approach is validated on both simulated and real data. Experimental results show that our approach is more accurate than existing state-of-the-art approaches, especially when dealing with low-textured objects with multiple coplanar faces.

show abstract

“…However, this approach presents a problem when considering the large range of module designs in circulation. A path planning framework is hence proposed to deal with this limitation in two stages: in the first, the module pose is identified through the use of a model-based tracking algorithm ( Trinh et al, 2018 ), given an approximate module pose is provided as input. In the second stage, point cloud data from an RGBD camera is used to directly define a tool path.…”

Section: Methodsmentioning

confidence: 99%

Vision-Guided MPC for Robotic Path Following Using Learned Memory-Augmented Model

Rastegarpanah

Hathaway

2021

Front. Robot. AI

View full text Add to dashboard Cite

The control of the interaction between the robot and environment, following a predefined geometric surface path with high accuracy, is a fundamental problem for contact-rich tasks such as machining, polishing, or grinding. Flexible path-following control presents numerous applications in emerging industry fields such as disassembly and recycling, where the control system must adapt to a range of dissimilar object classes, where the properties of the environment are uncertain. We present an end-to-end framework for trajectory-independent robotic path following for contact-rich tasks in the presence of parametric uncertainties. We formulate a combination of model predictive control with image-based path planning and real-time visual feedback, based on a learned state-space dynamic model. For modeling the dynamics of the robot-environment system during contact, we introduce the application of the differentiable neural computer, a type of memory augmented neural network (MANN). Although MANNs have been as yet unexplored in a control context, we demonstrate a reduction in RMS error of ∼21.0% compared with an equivalent Long Short-Term Memory (LSTM) architecture. Our framework was validated in simulation, demonstrating the ability to generalize to materials previously unseen in the training dataset.

show abstract

A modular framework for model-based visual tracking using edge, texture and depth features

Cited by 25 publications

References 27 publications

Tracking of Non-Rigid Objects using RGB-D Camera

Tracking of Non-Rigid Objects using RGB-D Camera

RGB-D Tracking of Complex Shapes Using Coarse Object Models

Vision-Guided MPC for Robotic Path Following Using Learned Memory-Augmented Model

Contact Info

Product

Resources

About