Registration of 3D CT Data to 2D Endoscopic Image using a Gradient Mutual Information based Viewpoint Matching for Image-Guided Medialization Laryngoplasty

Yim, Younghee; Wakid, Mike; Kirmizibayrak, Can; Bielamowicz, Steven; Hahn, James K.

doi:10.5626/jcse.2010.4.4.368

Cited by 9 publications

(5 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Cam. correction Osechinskiy et al [23] MR --Gill et al [12] MR --Dalvi et al [3] US --Zikic et al [40] FL --Pickering et al [24] FL --Estepar et al [6] US -Hernes et al [15] US -Yim et al [39] EN Su et al [31] EN Merritt et al [18] EN Prisacariu et al [27] NMP Sandhu et al [29] NMP Dambreville et al [4] NMP Prisacariu et al [26] NMP Nosrati et al [22] EN Proposed method EN quality, or information content of intraoperative X-ray and US still markedly lags behind the typically high resolution 3D preoperative data, and endoscopic imaging remains the staple modality in MIS. The current approach used in the operating room of mentally reconstructing locations of various structures during surgery by transferring the mental abstraction from 3D to 2D data, is an error-prone procedure especially if the surgeon's level of experience is limited.…”

Section: Methodsmentioning

confidence: 99%

“…Other works focused on feature tracking in which corresponding points on endoscopic video and preoperative data are assumed to be known [28]. While the registration in [25], [28], [34] is performed manually, the methods proposed in Yim et al [39] and Merritt et al [18] are able to automatically find the 3D pose of the objects and rigidly register 3D CT data to a 2D endoscopy image. None of the aforementioned methods can handle free-form deformation of tissues that usually happens due to respiratory motion and/or surgical intervention.…”

Section: A Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Simultaneous Multi-Structure Segmentation and 3D Nonrigid Pose Estimation in Image-Guided Robotic Surgery

Nosrati

Abugharbieh

Peyrat

et al. 2016

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

Abstract-In image-guided robotic surgery, labeling and segmenting the endoscopic video stream into meaningful parts provides important contextual information that surgeons can exploit to enhance their perception of the surgical scene. This information can provide surgeons with real-time decision-making guidance before initiating critical tasks such as tissue cutting. Segmenting endoscopic video is a very challenging problem due to a variety of complications including significant noise and clutter attributed to bleeding and smoke from cutting, poor color and texture contrast between different tissue types, occluding surgical tools, and limited (surface) visibility of the objects' geometries on the projected camera views. In this paper, we propose a multi-modal approach to segmentation where preoperative 3D computed tomography scans and intraoperative stereo-endoscopic video data are jointly analyzed. The idea is to segment multiple poorly visible structures in the stereo/multichannel endoscopic videos by fusing reliable prior knowledge captured from the preoperative 3D scans. More specifically, we estimate and track the pose of the preoperative models in 3D and consider the models' non-rigid deformations to match with corresponding visual cues in multi-channel endoscopic video and segment the objects of interest. Further, contrary to most augmented reality frameworks in endoscopic surgery that assume known camera parameters, an assumption that is often violated during surgery due to non-optimal camera calibration and changes in camera focus/zoom, our method embeds these parameters into the optimization hence correcting the calibration parameters within the segmentation process. We evaluate our technique in several scenarios: synthetic data, ex vivo lamb kidney datasets, and in vivo clinical partial nephrectomy surgery with results demonstrating high accuracy and robustness.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: A Related Workmentioning

confidence: 99%

Simultaneous Multi-Structure Segmentation and 3D Nonrigid Pose Estimation in Image-Guided Robotic Surgery

Nosrati

Abugharbieh

Peyrat

et al. 2016

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

show abstract

“…Many algorithms have been proposed specifically for endoscope-CT registration. Combinations of different intensity based schemes such as crosscorrelation, squared intensity difference, mutual information and pattern intensity have shown promising results [6,7]. Similarly, feature based schemes involving natural landmarks, contour based feature points, iterative closest point and k-means clustering have also been exploited [8,9].…”

Section: Introductionmentioning

confidence: 99%

Real-Time Augmented Reality for Ear Surgery

Hussain

Lalande

Marroquin

et al. 2018

Medical Image Computing and Computer Assisted Intervention – MICCAI 2018

View full text Add to dashboard Cite

Transtympanic procedures aim at accessing the middle ear structures through a puncture in the tympanic membrane. They require visualization of middle ear structures behind the eardrum. Up to now, this is provided by an oto endoscope. This work focused on implementing a real-time augmented reality based system for robotic-assisted transtympanic surgery. A preoperative computed tomography scan is combined with the surgical video of the tympanic membrane in order to visualize the ossciles and labyrinthine windows which are concealed behind the opaque tympanic membrane. The study was conducted on 5 artificial and 4 cadaveric temporal bones. Initially, a homography framework based on fiducials (6 stainless steel markers on the periphery of the tympanic membrane) was used to register a 3D reconstructed computed tomography image to the video images. Micro/endoscope movements were then tracked using Speeded-Up Robust Features. Simultaneously, a micro-surgical instrument (needle) in the frame was identified and tracked using a Kalman filter. Its 3D pose was also computed using a 3-collinear-point framework. An average initial registration accuracy of 0.21 mm was achieved with a slow propagation error during the 2-minute tracking. Similarly, a mean surgical instrument tip 3D pose estimation error of 0.33 mm was observed. This system is a crucial first step towards keyhole surgical approach to middle and inner ears.

show abstract

“…Indeed, in most computer-assisted surgical systems, image registration plays paramount role in the overall performance of the system. In endoscope-CT registration, combinations of different intensity-based schemes such as cross-correlation, squared intensity difference, pattern intensity, normalised and gradient mutual information have shown promising results 40,41 . Similarly, feature-based schemes involving natural landmarks, contour based feature points, iterative closest point and k-means clustering have also been exploited [42][43][44] .…”

Section: Discussionmentioning

confidence: 99%

Video-based augmented reality combining CT-scan and instrument position data to microscope view in middle ear surgery

Hussain¹,

Lalande²,

Marroquin³

et al. 2020

Sci Rep

View full text Add to dashboard Cite

the aim of the study was to develop and assess the performance of a video-based augmented reality system, combining preoperative computed tomography (ct) and real-time microscopic video, as the first crucial step to keyhole middle ear procedures through a tympanic membrane puncture. Six different artificial human temporal bones were included in this prospective study. Six stainless steel fiducial markers were glued on the periphery of the eardrum, and a high-resolution CT-scan of the temporal bone was obtained. Virtual endoscopy of the middle ear based on this CT-scan was conducted on Osirix software. Virtual endoscopy image was registered to the microscope-based video of the intact tympanic membrane based on fiducial markers and a homography transformation was applied during microscope movements. These movements were tracked using Speeded-Up Robust Features (SURF) method. Simultaneously, a micro-surgical instrument was identified and tracked using a Kalman filter. The 3D position of the instrument was extracted by solving a three-point perspective framework. For evaluation, the instrument was introduced through the tympanic membrane and ink droplets were injected on three middle ear structures. An average initial registration accuracy of 0.21 ± 0.10 mm (n = 3) was achieved with a slow propagation error during tracking (0.04 ± 0.07 mm). The estimated surgical instrument tip position error was 0.33 ± 0.22 mm. The target structures' localization accuracy was 0.52 ± 0.15 mm. The submillimetric accuracy of our system without tracker is compatible with ear surgery.

show abstract

Registration of 3D CT Data to 2D Endoscopic Image using a Gradient Mutual Information based Viewpoint Matching for Image-Guided Medialization Laryngoplasty

Cited by 9 publications

References 51 publications

Simultaneous Multi-Structure Segmentation and 3D Nonrigid Pose Estimation in Image-Guided Robotic Surgery

Simultaneous Multi-Structure Segmentation and 3D Nonrigid Pose Estimation in Image-Guided Robotic Surgery

Real-Time Augmented Reality for Ear Surgery

Video-based augmented reality combining CT-scan and instrument position data to microscope view in middle ear surgery

Contact Info

Product

Resources

About