Towards Occlusion-Free Surgical Instrument Tracking: A Modular Monocular Approach and an Agile Calibration Method

Registration that aligns different views of one interested organ together is an essential technique and outstanding problem in medical robotics and image-guided surgery (IGS). This work introduces a novel rigid point set registration (PSR) approach that aims to accurately map the pre-operative space with the intra-operative space to enable successful image guidance for computer-assisted orthopaedic surgery (CAOS). The normal vectors and tangent vectors are first extracted from the pre-operative and intra-operative point sets (PSs) respectively, and are further utilized to enhance the registration accuracy and robustness. The contributions of this article are threefolds. First, we propose and formulate a novel distribution that describes the error between one normal vector and the corresponding tangent vector based on the von-Mises Fisher (vMF) distribution. Second, by modelling the anisotropic position localization error with the multi-variate Gaussian distribution, we formulate the PSR considering anisotropic localization error as a maximum likelihood estimation (MLE) problem and then solve it under the expectation maximization (EM) framework. Third, to facilitate the optimization process, the gradients of the objective function with respect to the desired parameters are computed and presented. Extensive experimental results on the human femur and pelvis models verify that the proposed approach outperforms the state-of-the-art methods, and demonstrate potential clinical values for relevant surgical navigation applications.

show abstract

“…E. The Expression of ∂C P,mn ∂dt in (12) ∂C P,mn ∂dt = (Σ q−1 ) −1 − x n + dt + dR R q−1 y m + t q−1 .…”

Section: A Von-mises Fisher Distributionmentioning

confidence: 99%

Aligning 3D Curve With Surface Using Tangent and Normal Vectors for Computer-Assisted Orthopedic Surgery

Min

Zhu

Liu

et al. 2021

IEEE Trans. Med. Robot. Bionics

Self Cite

View full text Add to dashboard Cite

show abstract

“…In the recent work of Zhao et al [28], the authors use both 2D chessboard calibration patterns and squared fiducial marker attached to the cameras in order to obtain the extrinsic parameters. Weng et al [4] propose a non-iterative method to obtain the camera extrinsic parameter using a single fiducial squared marker.…”

Section: B Camera Calibrationmentioning

confidence: 99%

“…Square planar markers, comprised of a black external border and an inner binary pattern have gained popularity since their pose can be estimated by only using its four corners. As a consequence, they are being employed in 6DoF tracking tasks in different disciplines such as human-computer interaction [1], augmented/virtual reality [2], robotics [3] or in medicine for tracking of surgical instruments [4][5][6][7], a crucial process of image-guided surgery providing the position and orientation (pose) of the instruments with respect to the patient in preoperative registration and intraoperative navigation.…”

Section: Introductionmentioning

confidence: 99%

Simultaneous Multi-View Camera Pose Estimation and Object Tracking With Squared Planar Markers

Sarmadi¹,

Muñoz-Salinas²,

Berbís³

et al. 2019

IEEE Access

View full text Add to dashboard Cite

Object tracking is a key aspect in many applications, such as augmented reality in medicine (e.g., tracking a surgical instrument) or robotics. Squared planar markers have become popular tools for tracking since their pose can be estimated from their four corners. While using a single marker and a single camera limits the working area considerably, using multiple markers attached to an object requires estimating their relative position, which is not trivial, for high-accuracy tracking. Likewise, using multiple cameras requires estimating their extrinsic parameters, also a tedious process that must be repeated whenever a camera is moved. This paper proposes a novel method to simultaneously solve the above-mentioned problems. From a video sequence showing a rigid set of planar markers recorded from multiple cameras, the proposed method is able to automatically obtain the three-dimensional configuration of the markers, the extrinsic parameters of the cameras, and the relative pose between the markers and the cameras at each frame. Our experiments show that our approach can obtain highly accurate results for estimating these parameters using the low-resolution cameras. Once the parameters are obtained, tracking of the object can be done in real time with a low computational cost. The proposed method is a step forward in the development of cost-effective solutions for object tracking. INDEX TERMS Fiducial markers, camera pose estimation, robotics, augmented reality.

show abstract

“…As can be seen in Figure 1h, ePnP has a larger workspace, but the depth accuracy is inferior because the image coordinates of the features are insensitive to depth displacement. To offset the depth accuracy problem of ePnP, some researchers developed systems using multiple cameras (multi-view system) [17,22,23]. In these systems, every camera produces a pose estimation by ePnP.…”

Section: Introductionmentioning

confidence: 99%

Distributed visual positioning for surgical instrument tracking

Cai,

Zhu,

et al. 2024

Phys Eng Sci Med

View full text Add to dashboard Cite

In clinical operations, it is crucial for surgeons to know the location of the surgical instrument. Traditional positioning systems have difficulty dealing with camera occlusion, marker occlusion, and environmental interference. To address these issues, we propose a distributed visual positioning system for surgical instrument tracking in surgery. First, we design the marker pattern with a black and white triangular grid and dot that can be adapted to various instrument surfaces and improve the marker location accuracy of the feature. The cross-points in the marker are the features that each feature has a unique ID. Furthermore, we proposed detection and identification for the position-sensing marker to realize the accurate location and identification of features. Second, we introduce mPnP (multi Perspective-n-Point) method, which fuses feature coordinates from all cameras to deduce the final result directly by the intrinsic and extrinsic parameters. This method provides a reliable initial value for the Bundle Adjustment algorithms. During instrument tracking, we assess the motion state of the instrument and select either dynamic or static Kalman filtering to mitigate any jitter in the instrument's movement. The core algorithms comparison experiment indicates our positioning algorithm has a lower reprojection error comparison to the mainstream algorithms. A series of quantitative experiments showed that the proposed system positioning error is below 0.207 mm, and the run time is below 118.842 ms. The results demonstrate 1 Manuscript the tremendous clinical application potential of our system providing accurate positioning of instruments promoting the efficiency and safety of clinical surgery.

show abstract

Towards Occlusion-Free Surgical Instrument Tracking: A Modular Monocular Approach and an Agile Calibration Method

Cited by 35 publications

References 35 publications

Aligning 3D Curve With Surface Using Tangent and Normal Vectors for Computer-Assisted Orthopedic Surgery

Aligning 3D Curve With Surface Using Tangent and Normal Vectors for Computer-Assisted Orthopedic Surgery

Simultaneous Multi-View Camera Pose Estimation and Object Tracking With Squared Planar Markers

Distributed visual positioning for surgical instrument tracking

Contact Info

Product

Resources

About