Hand-eye calibration for robotic assisted minimally invasive surgery without a calibration object

Pachtrachai, Krittin; Allan, Max; Pawar, Vijay; Hailes, Stephen; Stoyanov, Danail

doi:10.1109/iros.2016.7759387

Cited by 39 publications

(20 citation statements)

References 43 publications

(59 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…23 The fetoscope was precalibrated using the Matlab Camera Calibration Toolbox. We also precomputed and applied the Hand-eye Calibration 24 matrix from a sequence of images of a checkerboard as well as synchronized sensor poses. The window size of the superframe was determined from the trade-off analysis as W ¼ 5, which is presented later in Sec.…”

Section: Metrics and Implementation Detailsmentioning

confidence: 99%

Pruning strategies for efficient online globally consistent mosaicking in fetoscopy

et al. 2019

Self Cite

View full text Add to dashboard Cite

Twin-to-twin transfusion syndrome is a condition in which identical twins share a certain pattern of vascular connections in the placenta. This leads to an imbalance in the blood flow that, if not treated, may result in a fatal outcome for both twins. To treat this condition, a surgeon explores the placenta with a fetoscope to find and photocoagulate all intertwin vascular connections. However, the reduced field of view of the fetoscope complicates their localization and general overview. A much more effective exploration could be achieved with an online mosaic created at exploration time. Currently, accurate, globally consistent algorithms such as bundle adjustment cannot be used due to their offline nature, while online algorithms lack sufficient accuracy. We introduce two pruning strategies facilitating the use of bundle adjustment in a sequential fashion: (1) a technique that efficiently exploits the potential of using an electromagnetic tracking system to avoid unnecessary matching attempts between spatially inconsistent image pairs, and (2) an aggregated representation of images, which we refer to as superframes, that allows decreasing the computational complexity of a globally consistent approach. Quantitative and qualitative results on synthetic and phantom-based datasets demonstrate a better trade-off between efficiency and accuracy.

show abstract

Section: Metrics and Implementation Detailsmentioning

confidence: 99%

Pruning strategies for efficient online globally consistent mosaicking in fetoscopy

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…Previous work on the hand-eye problem has also shown that hand-eye calibration of a da Vinci Surgical robot yields significantly larger errors than with other robots mainly due to the strict RCM constraints of the robot motion [13]. While this problem can be attenuated using an EndoWrist surgical tool as a calibration target instead of a static checkerboard [12], [31] and given that it uses RCM constraints, this still heavily relies on accurate visual tool tracking, and thus introduces an additional source of calibration error.…”

Section: Related Workmentioning

confidence: 99%

Hand-Eye Calibration With a Remote Centre of Motion

Pachtrachai

Vasconcelos

Dwyer

et al. 2019

IEEE Robot. Autom. Lett.

Self Cite

View full text Add to dashboard Cite

In the eye-in-hand robot configuration, hand-eye calibration plays a vital role in completing the link between the robot and camera coordinate systems. Calibration algorithms are mature and provide accurate transformation estimations for an effective camera-robot link but rely on a sufficiently wide range of calibration data to avoid errors and degenerate configurations. This can be difficult in the context of keyhole surgical robots because they are mechanically constrained to move around a remote centre of motion (RCM) which is located at the trocar port. The trocar limits the range of feasible calibration poses that can be obtained and results in ill-conditioned hand-eye constraints. In this paper, we propose a new approach to deal with this problem by incorporating the RCM constraints into the hand-eye formulation. We show that this not only avoids illconditioned constraints but is also more accurate than classic hand-eye calibration with a free 6DoF motion, due to solving simpler equations that take advantage of the reduced DoF. We validate our method using simulation to test numerical stability and a physical implementation on an RCM constrained KUKA LBR iiwa 14 R820 equipped with a NanEye stereo camera.

show abstract

“…The algorithm we propose in this letter consists of synchronisation using cross-correlation to estimate the time-delay between two data streams and then data recovery using a method based on screw constraints for the lower-sampling rate data. The resynchronised data streams are then fed into the hand-eye calibration algorithm developed in [13] to finally determine the hand-eye transformation.…”

Section: Hand-eye Calibration Using Unsynchronised Datamentioning

confidence: 99%

“…Several formulations on different domain representations can be developed to tackle the problem e.g. Euclidean group [7], [9], [10], quaternion [11]- [13] and dual quaternion [14]- [16]. Constraints for the calibration optimisation can be developed, such as using epipolar constraints [17], using pure rotational movements [18], performing handeye calibration for fewer unknown parameters for SCARA robot [19] or using global optimisation methods [20].…”

mentioning

confidence: 99%

CHESS—Calibrating the Hand-Eye Matrix With Screw Constraints and Synchronization

Pachtrachai

Vasconcelos

Dwyer

et al. 2018

IEEE Robot. Autom. Lett.

Self Cite

View full text Add to dashboard Cite

Hand-eye calibration is a classic problem in robotics that aims to find the transformation between two rigidly attached reference frames, usually a camera and a robot end-effector or a motion tracker. Most hand-eye calibration techniques require two data streams, one containing the eye (camera) motion and the other containing the hand (robot/tracker) motion, and the classic hand-eye formulation assumes that both data streams are fully synchronized. However, different motion capturing devices and cameras often have variable capture rates and timestamps that cannot always be easily triggered in sync. Although probabilistic approaches have been proposed to solve for nonsynchronized data streams, they are not able to deal with different capture rates. We propose a new approach for unsynchronized hand-eye calibration that is able to deal with different capture rates and time delays. Our method interpolates and resamples the signal with the lowest capture rate in a way that is consistent with the twist motion constraints of the hand-eye problem. Cross-correlation techniques are then used to produce two fully synchronized data streams that can be used to solve the hand-eye problem with classic methods. In our experimental section, we show promising validation results on simulation data and also on real data obtained from a robotic arm holding a camera. Index Terms-Calibration and identification, kinematics. I. INTRODUCTION S ENSORS and vision systems are often integrated into robotic platforms to provide extra information of the surroundings which helps in localisation of the workspace [1]. An example in robotics-assisted minimally invasive surgery (RMIS) is stereo-laparoscopic cameras or GPS in mobile vehicles for various types of simultaneous localization and mapping (SLAM) applications [2]. For RMIS, accurate and real-time localisation of an operative site with a laparoscope is an important component towards developing new capabilities in robotic surgery such as autonomy with visual servoing or surgical

show abstract

Hand-eye calibration for robotic assisted minimally invasive surgery without a calibration object

Cited by 39 publications

References 43 publications

Pruning strategies for efficient online globally consistent mosaicking in fetoscopy

Pruning strategies for efficient online globally consistent mosaicking in fetoscopy

Hand-Eye Calibration With a Remote Centre of Motion

CHESS—Calibrating the Hand-Eye Matrix With Screw Constraints and Synchronization

Contact Info

Product

Resources

About