CHESS—Calibrating the Hand-Eye Matrix With Screw Constraints and Synchronization

Abstract

Hand-eye calibration is a classic problem in robotics that aims to find the transformation between two rigidly attached reference frames, usually a camera and a robot end-effector or a motion tracker. Most hand-eye calibration techniques require two data streams, one containing the eye (camera) motion and the other containing the hand (robot/tracker) motion, and the classic hand-eye formulation assumes that both data streams are fully synchronized. However, different motion capturing devices and cameras often have variable capture rates and timestamps that cannot always be easily triggered in sync. Although probabilistic approaches have been proposed to solve for nonsynchronized data streams, they are not able to deal with different capture rates. We propose a new approach for unsynchronized hand-eye calibration that is able to deal with different capture rates and time delays. Our method interpolates and resamples the signal with the lowest capture rate in a way that is consistent with the twist motion constraints of the hand-eye problem. Cross-correlation techniques are then used to produce two fully synchronized data streams that can be used to solve the hand-eye problem with classic methods. In our experimental section, we show promising validation results on simulation data and also on real data obtained from a robotic arm holding a camera. Index Terms-Calibration and identification, kinematics. I. INTRODUCTION S ENSORS and vision systems are often integrated into robotic platforms to provide extra information of the surroundings which helps in localisation of the workspace [1]. An example in robotics-assisted minimally invasive surgery (RMIS) is stereo-laparoscopic cameras or GPS in mobile vehicles for various types of simultaneous localization and mapping (SLAM) applications [2]. For RMIS, accurate and real-time localisation of an operative site with a laparoscope is an important component towards developing new capabilities in robotic surgery such as autonomy with visual servoing or surgical

Augmented reality needle ablation guidance tool for irreversible electroporation in the pancreas

“…Since IR tracking provides precise position of the US probe relative to the HoloLens headset, the data extracted from US can be used to further increase the accuracy of visual guides. Another additional feature would be to sync and calibrate data together from the US and the HoloLens 13 .…”

A Comparative Review of Hand-Eye Calibration Techniques for Vision Guided Robots

“…By timestamping the data from both sources, users could manually or programmatically synchronise the data streams and also avoid missing data. Cross-correlation techniques can also been used to achieve data synchronisation for hand-eye calibration as in [107], [108]. Normalising and resampling the data before the crosscorrelation operation can be used to ensure that differential data length caused by time delay or different sampling rate do not affect the result.…”

Robust Task-Oriented Markerless Extrinsic Calibration for Robotic Pick-and-Place Scenarios

“…But they need an additional contact sensor to get contact information. Pachtrachai et al [18] solved unsynchronized hand-eye calibration that the data streams are different capture rates and time delays. They use cross-correlation to synchronize data and use screw constrains to recover data.…”