Three-dimensional visual tracking and pose estimation in Scanning Electron Microscopes

Cui, Le; Marchand, Éric; Haliyo, Sinan; Régnier, Stéphane

doi:10.1109/iros.2016.7759766

Cited by 11 publications

(8 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…End-users would never see an out-of-focus image, but instead of a live-feed, the system would freeze frequently. Cui, Marchand, Haliyo, and Régnier [10] could focus on moving objects, but only when they were provided with enough training data beforehand. Once more, this was not a feasible approach for this system as almost anything could be put into the field of view of the camera, and training was not possible.…”

Section: Related Workmentioning

confidence: 99%

Validating Autofocus Algorithms with Automated Tests

Werner

Carrasco²

2018

Robotics

View full text Add to dashboard Cite

Abstract:For an automated camera focus, a fast and reliable algorithm is key to its success. It should work in a precisely defined way for as many cases as possible. However, there are many parameters which have to be fine-tuned for it to work exactly as intended. Most literature only focuses on the algorithm itself and tests it with simulations or renderings, but not in real settings. Trying to gather this data by manually placing objects in front of the camera is not feasible, as no human can perform one movement repeatedly in the same way, which makes an objective comparison impossible. We therefore used a small industrial robot with a set of over 250 combinations of movement, pattern, and zoom-states to conduct these tests. The benefit of this method was the objectivity of the data and the monitoring of the important thresholds. Our interest laid in the optimization of an existing algorithm, by showing its performance in as many benchmarks as possible. This included standard use cases and worst-case scenarios. To validate our method, we gathered data from a first run, adapted the algorithm, and conducted the tests again. The second run showed improved performance.

show abstract

Section: Related Workmentioning

confidence: 99%

Validating Autofocus Algorithms with Automated Tests

Werner

Carrasco²

2018

Robotics

View full text Add to dashboard Cite

show abstract

“…Prior research on 3D tracking used multi-cameras [21, 22] and depth-from-focus techniques to estimate the 3D position of the miniaturized agents [23, 24]. Recently, a template-based hybrid visual tracking algorithm was presented to estimate the 3D posture of micro-objects in a scanning electron microscope [25]. The algorithm used luminance information to estimate the object displacement on the x - y plane and its orientation around the z -axis.…”

Section: Introductionmentioning

confidence: 99%

A GPU-accelerated model-based tracker for untethered submillimeter grippers

Scheggi

Yoon

Ghosh

et al. 2018

Robotics and Autonomous Systems

View full text Add to dashboard Cite

Miniaturized grippers that possess an untethered structure are suitable for a wide range of tasks, ranging from micromanipulation and microassembly to minimally invasive surgical interventions. In order to robustly perform such tasks, it is critical to properly estimate their overall configuration. Previous studies on tracking and control of miniaturized agents estimated mainly their 2D pixel position, mostly using cameras and optical images as a feedback modality. This paper presents a novel solution to the problem of estimating and tracking the 3D position, orientation and configuration of the tips of submillimeter grippers from marker-less visual observations. We consider this as an optimization problem, which is solved using a variant of the Particle Swarm Optimization algorithm. The proposed approach has been implemented in a Graphics Processing Unit (GPU) which allows a user to track the submillimeter agents online. The proposed approach has been evaluated on several image sequences obtained from a camera and on B-mode ultrasound images obtained from an ultrasound probe. The sequences show the grippers moving, rotating, opening/closing and grasping biological material. Qualitative results obtained using both hydrogel (soft) and metallic (hard) grippers with different shapes and sizes ranging from 750 microns to 4 mm (tip to tip), demonstrate the capability of the proposed method to track the agent in all the video sequences. Quantitative results obtained by processing synthetic data reveal a tracking position error of 25 ± 7 μm and orientation error of 1.7 ± 1.3 degrees. We believe that the proposed technique can be applied to different stimuli responsive miniaturized agents, allowing the user to estimate the full configuration of complex agents from visual marker-less observations.

show abstract

“…The first group of motion estimation techniques is based on pose computation. In [6], positions along x-(t x ) and y-axis (t y ), and rotation about an optical axis (R z ) were computed using Gauss-Newton method by minimizing the sum of the projection errors of some points of a pre-defined 2D model. Authors assumed that the object, while moving, stays on the plane parallel to the image plane.…”

Section: Introductionmentioning

confidence: 99%

“…In [9] and [10], cross-correlation was used, whereas in [11] model was represented by active contours and motion was estimated from the minimization of the active contours and the object detected edges. Both methods have the same drawback of [6]: as the rotations R x and R y are not considered, measured quantities may be inaccurate. In [12] an interesting method was proposed, it is based on the work described in [13] which used spherical Fourier transform to compute the rotations R x , R y and R z .…”

Section: Introductionmentioning

confidence: 99%

Full 3D rotation estimation in scanning electron microscope

Kudryavtsev

Dembélé

Piat

2017

2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Estimation of 3D object position is a crucial step for a variety of robotics and computer vision applications including 3D reconstruction and object manipulation. When working in microscale, new types of visual sensors are used such as Scanning Electron Microscope (SEM). Nowadays, microand nanomanipulation tasks, namely components assembly, are performed in teleoperated mode in most of the cases. Measuring object position and orientation is a crucial step towards automatic object handling. Current methods of pose estimation in SEM allow recovering full object movement using its computer-aided design (CAD) model. If the model is not known, most methods allow to estimate only in-plane translations and rotation around camera optical axis. In the literature, SEM is considered as a camera with parallel projection or an affine camera, which means image invariance to z-movement and bas-relief ambiguity. In this paper, authors address the problem of measuring full 3D rotation of the unknown scene for uncalibrated SEM without additional sensors. Rotations are estimated from image triplets by solving a spherical triangle from fundamental matrices only, without need of intrinsic calibration, allowing to avoid parallel projection ambiguities. The presented results, obtained in simulation and on real data, allow validating the proposed scheme.

show abstract

Three-dimensional visual tracking and pose estimation in Scanning Electron Microscopes

Cited by 11 publications

References 16 publications

Validating Autofocus Algorithms with Automated Tests

Validating Autofocus Algorithms with Automated Tests

A GPU-accelerated model-based tracker for untethered submillimeter grippers

Full 3D rotation estimation in scanning electron microscope

Contact Info

Product

Resources

About