Simultaneous Tracking, 3D Reconstruction and Deforming Point Detection for Stereoscope Guided Surgery

Lin, Bingxiong; Johnson, Adrian; Qian, Xiaoning; Sánchez, J.; Sun, Yu

doi:10.1007/978-3-642-40843-4_5

Cited by 35 publications

(34 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Another special property of MIS images is the abundant specular reflections that are view-dependent and, therefore, can cause error to endoscope tracking if they are picked up as feature points. Similar to [2], [8], the specular reflections are detected as the pixels whose intensities are larger than a global threshold. In addition, their 3-by-3 neighbors are also marked as specular reflections.…”

Section: A Detecting Candidate Branching Pointsmentioning

confidence: 99%

See 1 more Smart Citation

Efficient Vessel Feature Detection for Endoscopic Image Analysis

Lin

Sun

Sánchez

et al. 2015

IEEE Trans. Biomed. Eng.

Self Cite

View full text Add to dashboard Cite

Distinctive feature detection is an essential task in computer-assisted minimally invasive surgery (MIS). For special conditions in an MIS imaging environment, such as specular reflections and texture homogeneous areas, the feature points extracted by general feature point detectors are less distinctive and repeatable in MIS images. We observe that abundant blood vessels are available on tissue surfaces and can be extracted as a new set of image features. In this paper, two types of blood vessel features are proposed for endoscopic images: branching points and branching segments. Two novel methods, ridgeness-based circle test and ridgeness-based branching segment detection are presented to extract branching points and branching segments, respectively. Extensive in vivo experiments were conducted to evaluate the performance of the proposed methods and compare them with the state-of-the-art methods. The numerical results verify that, in MIS images, the blood vessel features can produce a large number of points.More importantly, those points are more robust and repeatable than the other types of feature points. In addition, due to the difference in feature types, vessel features can be combined with other general features, which makes them new tools for MIS image analysis. These proposed methods are efficient and the code and datasets are made available to the public.

show abstract

Section: A Detecting Candidate Branching Pointsmentioning

confidence: 99%

“…The extracted image features can be used for tissue tracking [1]- [3], deformation recovery [4], [5], 3D reconstruction [6], [7], endoscope localization [8]- [10], augmented reality [8], [10], [11], and intra-operative registration [3], [12].…”

Section: Introductionmentioning

confidence: 99%

Efficient Vessel Feature Detection for Endoscopic Image Analysis

Lin

Sun

Sánchez

et al. 2015

IEEE Trans. Biomed. Eng.

Self Cite

View full text Add to dashboard Cite

show abstract

“…α jk ||A A A j j j (g g g k k k − g g g j j j ) + g g g j j j +t t t j j j − (g g g k k k +t t t k k k )|| 2 (7) where α jk is the weight calculated by the Euclidean distance of the two ED nodes. In [18], α jk is uniformly set to 1.…”

Section: E Energy Functionmentioning

confidence: 99%

“…In practice we found out that the extracted dense key points range from 100 to 1000 while conventional SIFT and SURF generate points from 0 to 200. After dense SURF process we refine key points and generate rigid rotation and translation by RANSAC which is a typical strategy adopted in implementing SLAM in MIS [6] [7]. Threshold for filtering outliers is set as 2 mm similar to [7].…”

Section: A Dense Surf Key Points Estimationmentioning

confidence: 99%

“…[4] and [5] extract whole tissue surface of organs from stereo or monocular images. [6] and [7] adopt conventional Simultaneous Localization and Mapping (SLAM) approaches, extended Kalman filter (EKF) SLAM and Parallel Tracking and Mapping, with modifications on separating rigid points and non-rigid points for tracking and sparse mapping. Contrary to feature extraction based methods, Du et al [8] employed an optical flow based approach namely Deformable Lucas-Kanade for tracking tissue surface.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Dynamic Reconstruction of Deformable Soft-Tissue With Stereo Scope in Minimal Invasive Surgery

Song

Wang

Zhao

et al. 2018

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

In minimal invasive surgery, it is important to rebuild and visualize the latest deformed shape of soft-tissue surfaces to mitigate tissue damages. This paper proposes an innovative Simultaneous Localization and Mapping (SLAM) algorithm for deformable dense reconstruction of surfaces using a sequence of images from a stereoscope. We introduce a warping field based on the Embedded Deformation (ED) nodes with 3D shapes recovered from consecutive pairs of stereo images. The warping field is estimated by deforming the last updated model to the current live model. Our SLAM system can: (1) Incrementally build a live model by progressively fusing new observations with vivid accurate texture. (2) Estimate the deformed shape of unobserved region with the principle As-Rigid-As-Possible. (3) Show the consecutive shape of models.(4) Estimate the current relative pose between the soft-tissue and the scope. In-vivo experiments with publicly available datasets demonstrate that the 3D models can be incrementally built for different soft-tissues with different deformations from sequences of stereo images obtained by laparoscopes. Results show the potential clinical application of our SLAM system for providing surgeon useful shape and texture information in minimal invasive surgery.

show abstract