Full 3D rotation estimation in scanning electron microscope

Kudryavtsev, Andrey V.; Dembélé, Sounkalo; Piat, Nadine

doi:10.1109/iros.2017.8202284

Cited by 9 publications

(7 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Based on the prior geometric knowledge of the target, the CAD model-based method is reliable to precisely track 6-DOF motions of rigid body targets [151], [172], as shown in Figure 17(b). Without the CAD model, one can still estimate the rotations by the geometric approach [178] or Fourier-based approach [45], [150]. Customized micro-vision-based systems are also developed for 3D motion measurement as well.…”

Section: B Multi-dof Capabilitymentioning

confidence: 99%

A Review of Computer Microvision-Based Precision Motion Measurement: Principles, Characteristics, and Applications

Sheng

Pang

et al. 2021

IEEE Trans. Instrum. Meas.

View full text Add to dashboard Cite

Micro/nano engineering is an emerging field which enables engineering and scientific discoveries in the microworld. As an effective and powerful tool for automation and manipulation at small scales, precision motion measurement by computer micro-vision is now broadly accepted and widely used in micro/nano engineering. Unlike other measurement methods, the vision-based techniques can intuitively visualize the measuring process with high interactivity, expansibility, and flexibility. This review paper aims to comprehensively present a survey of micro-vision-based motion measurement from the collective experience. Working principles of micro-vision systems are firstly introduced and described, where the hardware configuration, model calibration, and motion measurement algorithms are systematically summarized. The characteristics and performances of different micro-vision-based methods are then analyzed and discussed in terms of measurement resolution, range, degree of freedom, efficiency, and error sources. Recent advances of applications empowered by the developed computer micro-visionbased methods are also presented. The review can be helpful to researchers who engage in the development of micro-vision-based techniques, as well as provide the recent state and tendency for the research community of vision-based measurement, manipulation, and automation at micro/nano scales.

show abstract

Section: B Multi-dof Capabilitymentioning

confidence: 99%

A Review of Computer Microvision-Based Precision Motion Measurement: Principles, Characteristics, and Applications

Sheng

Pang

et al. 2021

IEEE Trans. Instrum. Meas.

View full text Add to dashboard Cite

show abstract

“…With three or more views, this ambiguity disappears in general. In our implementation, we use the self-calibration approach of [25] which computes directly, from the affine factorization of three views, the scene (structure and pose of the three views), up to an Euclidean transformation, as well as the intrinsic parameters. We use this to initialize a global non-linear optimization of all variables, as described in the following.…”

Section: Cost Function Formulationmentioning

confidence: 99%

“…In current cost function (25), all parameters have the same weight. However, we would like to tell the algorithm that the values of intrinsic parameters should be close to the initial ones and that the biggest impact should be made on the values of out-of-plane rotations.…”

Section: Regularizationmentioning

confidence: 99%

Autocalibration method for scanning electron microscope using affine camera model

Kudryavtsev

Guelpa

Rougeot

et al. 2020

Machine Vision and Applications

Self Cite

View full text Add to dashboard Cite

This paper deals with the task of autocalibration of Scanning Electron Microscope (SEM), which is a technique allowing to compute camera motion and intrinsic parameters. In contrast to classical calibration, which implies the use of a calibration object and is known to be a tedious and rigid operation, auto-or selfcalibration is performed directly on the images acquired for the visual task. As autocalibration represents an optimization problem, all the steps contributing to the success of the algorithm are presented: formulation of the cost function incorporating metric constraints, definition of bounds, regularization, and optimization algorithm. The presented method allows full estimation of camera matrices for all views in the sequence. It was validated on virtual images as well as on real SEM images (pollen grains, cutting tools, etc.). The results show a good convergence range and low execution time, notably compared to classical methods, and even more in the context of the calibration of SEM.

show abstract

“…15 In microrobotics research, feature based depth estimation for micro-grippers/micro-robots with Scanning Electron Microscope (SEM) [16][17][18] and 3D orientation estimation of the microrobot have been investigated. 19 Modelbased tracking of magnetic intraocular microrobot has also been proposed. 20 However, existing methods are often designed for a specific experimental setup and they cannot be adapted to different micro-robotic systems, such as mobile microrobots with complex shape and have various poses.…”

Section: Introductionmentioning

confidence: 99%

Data-Driven Microscopic Pose and Depth Estimation for Optical Microrobot Manipulation

Zhang

Zheng

et al. 2020

ACS Photonics

View full text Add to dashboard Cite

Optical microrobots have a wide range of applications in biomedical research for both in vitro and in vivo studies. In most microrobotic systems, the video captured by a monocular camera is the only way for capturing the movements of microrobots, and only planar motion can be captured by a monocular camera system. Accurate depth estimation is essential for the 3D reconstruction or autofocusing of micro-platforms, while the pose and depth estimation are necessary to enhance the 3D perception of the micro-robotic systems to enable dexterous micro-manipulation and other tasks. In this paper, we propose a data-driven method for pose and depth estimation in an optically manipulated microrobotic system. Focus measurement is used to obtain features for Gaussian Process Regression (GPR), which enables precise depth estimation. For mobile microrobots with varying poses, a novel method is developed based on a deep residual neural network with the incorporation of prior domain knowledge about the optical microrobots encoded via GPR. The method can simultaneously 1 track microrobots with complex shapes and estimate the pose and depth values of the optical microrobots.Cross-validation has been conducted to demonstrate the sub-micron accuracy of the proposed method and precise pose and depth perception for microrobots. We further demonstrate the generalisability of the method by adapting it to microrobots of different shapes using transfer learning with few-shot calibration. Intuitive visualization is provided to facilitate effective human-robot interaction during micro-manipulation based on pose and depth estimation results.

show abstract

Full 3D rotation estimation in scanning electron microscope

Cited by 9 publications

References 16 publications

A Review of Computer Microvision-Based Precision Motion Measurement: Principles, Characteristics, and Applications

A Review of Computer Microvision-Based Precision Motion Measurement: Principles, Characteristics, and Applications

Autocalibration method for scanning electron microscope using affine camera model

Data-Driven Microscopic Pose and Depth Estimation for Optical Microrobot Manipulation

Contact Info

Product

Resources

About