Ultrasound needle segmentation and trajectory prediction using excitation network

Lee, Jia Yi; Islam, Mobarakol; Woh, Jing Ru; Washeem, T. S. Mohamed; Ngoh, Clara Lee Ying; Wong, Weng Fai; Ren, Hongliang

doi:10.1007/s11548-019-02113-x

Cited by 36 publications

(38 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our algorithm demonstrated much poorer performance in kidney images than the other applications in this study; however, the DSC of 58.0% prior to postprocessing is similar to the value of 56.65% reported by Lee et al 37 Additionally, our median angular error of 2.9°for the kidney ablation tools may offer a substantial improvement over the angular error of 13.3°reported by Lee et al, 37 though a direct comparison is not possible given the use of RMS error in that study. Assuming a similar pixel size, our tip error appears larger than the distance error from the Lee et al 37 study; however, their value does not represent the tip error, which is often much higher since the largest segmentation challenge is typically the early truncation of the tool path in the insertion direction. The high errors reported by both studies on the localization of tools in kidney images shows the US visibility challenges associated with steep tool insertion angles and increased number of anatomical interfaces with similar echogenicity, emphasizing the need for further research in this area.…”

Section: Discussionsupporting

confidence: 68%

“…The simpler approach to tool fitting was implemented using a largest island postprocessing technique to save only the largest connected region of predicted pixels, as has been previously studied. 37 A linear least-squares fit on this map was then used to predict the tool's tip and trajectory. In an attempt to establish the tool axis more robustly in the presence of disconnected outlier regions, a RANSAC model-fitting approach 12 was also evaluated.…”

Section: C Postprocessing and Evaluationmentioning

confidence: 99%

“…This method used images acquired at three frames per second from eight patient US videos to create a dataset for training and testing, achieving a Dice similarity coefficient (DSC) of 56.65%, root-mean-square (RMS) distance error of 9.5 pixels, and RMS angular error of 13.3°. 37 Although this approach is susceptible to information leakage since many images were acquired per patient, this study demonstrates many of the challenges associated with localizing needles in kidney images and the extension of this method to applications other than kidney biopsy has not been explored. Mwikirize et al 30 investigated an algorithm using a fully convolutional network with a fast region-based CNN for interventional applications requiring quick localization of mid-to-steeply inserted needles (40-75°) and deep insertions (up to 9 cm) in 2D US images.…”

Section: Introductionmentioning

confidence: 99%

“…[29][30][31][32][33][34][35][36][37][38] Additionally, approaches leveraging the principles of transfer learning (applying models developed for a prior task to a new context) have been investigated in the broader context of image segmentation, including a recent study in x-ray computed tomography (CT) images by Jung et al 39 In this paper, Jung et al 39 investigated the generalizability of a CNN-based method developed on a tandem-andovoids intracavitary brachytherapy device to segment other devices used for this type of procedure, substantiating the feasibility of a generalized tool segmentation method. Some existing works have specifically investigated CNN-based approaches to tool segmentation in US images, [29][30][31]34,37,38 recognizing that the appearance of tools in this imaging modality introduces unique challenges. Recent developments by Pourtaherian et al, 31 Arif et al, 29 and Zhang et al 38 focused on needle detection in 3D US images.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Deep learning segmentation of general interventional tools in two‐dimensional ultrasound images

et al. 2020

View full text Add to dashboard Cite

Purpose: Many interventional procedures require the precise placement of needles or therapy applicators (tools) to correctly achieve planned targets for optimal diagnosis or treatment of cancer, typically leveraging the temporal resolution of ultrasound (US) to provide real-time feedback. Identifying tools in two-dimensional (2D) images can often be time-consuming with the precise position difficult to distinguish. We have developed and implemented a deep learning method to segment tools in 2D US images in near real-time for multiple anatomical sites, despite the widely varying appearances across interventional applications. Methods: A U-Net architecture with a Dice similarity coefficient (DSC) loss function was used to perform segmentation on input images resized to 256 × 256 pixels. The U-Net was modified by adding 50% dropouts and the use of transpose convolutions in the decoder section of the network. The proposed approach was trained with 917 images and manual segmentations from prostate/gynecologic brachytherapy, liver ablation, and kidney biopsy/ablation procedures, as well as phantom experiments. Real-time data augmentation was applied to improve generalizability and doubled the dataset for each epoch. Postprocessing to identify the tool tip and trajectory was performed using two different approaches, comparing the largest island with a linear fit to random sample consensus (RAN-SAC) fitting. Results: Comparing predictions from 315 unseen test images to manual segmentations, the overall median [first quartile, third quartile] tip error, angular error, and DSC were 3.5 [1.3, 13.5] mm, 0.8 [0.3, 1.7]°, and 73.3 [56.2, 82.3]%, respectively, following RANSAC postprocessing. The predictions with the lowest median tip and angular errors were observed in the gynecologic images (median tip error: 0.3 mm; median angular error: 0.4°) with the highest errors in the kidney images (median tip error: 10.1 mm; median angular error: 2.9°). The performance on the kidney images was likely due to a reduction in acoustic signal associated with oblique insertions relative to the US probe and the increased number of anatomical interfaces with similar echogenicity. Unprocessed segmentations were performed with a mean time of approximately 50 ms per image. Conclusions: We have demonstrated that our proposed approach can accurately segment tools in 2D US images from multiple anatomical locations and a variety of clinical interventional procedures in near real-time, providing the potential to improve image guidance during a broad range of diagnostic and therapeutic cancer interventions.

show abstract

Section: Discussionsupporting

confidence: 68%

Section: C Postprocessing and Evaluationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Deep learning segmentation of general interventional tools in two‐dimensional ultrasound images

et al. 2020

View full text Add to dashboard Cite

show abstract

“…The initial trajectory region is analyzed with ranklet and Radon transforms. Lee et al [27] used neural networks and spatial information to segment and track the needle tip. Xu et al [28] proposed a pipeline that includes thresholding, morphological operations, and maximum likelihood estimation sample consensus algorithm.…”

Section: Related Studies In the Literaturementioning

confidence: 99%

Gabor filter-based localization of straight and curved needles in 2D ultrasound images

Kaya¹,

Senel²,

Bebek³

2020

Turk J Elec Eng & Comp Sci

View full text Add to dashboard Cite

2D ultrasound (US) is one of the most commonly used medical imaging devices for needle localization in biopsies. However, the produced images are low-resolution and contain an excessive number of artifacts, which makes the needle localization challenging. Image processing techniques can help resolve this issue. This paper presents a novel Gabor filter-based method for needle localization in 2D US images, which enhances the needle outline in the images while suppressing other structures. The scheme works in two stages: First, the Gabor filter is applied to the image, the needle insertion angle is estimated, and the needle trajectory is found using a RANSAC line fitting; then, the Gabor filter is repeated using the estimated insertion angle and the location of the needle tip is estimated using probability mapping. The proposed scheme works with both straight and curved needles. An automatic parameter tuning method, which is used to optimize the threshold value of the Otsu's method, is also presented here. The tests on this needle localization scheme were done using two different water mixtures, three different gelatin-based phantoms, and ex vivo experiments. The accuracy was assessed using an infrared-camera-based tracking sensor and computed tomography images. The results showed that the suggested localization scheme can be effectively used in the 2D US image-guided needle procedures.

show abstract

Learning and Reasoning with the Graph Structure Representation in Robotic Surgery

Islam

Seenivasan

Ming

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Ultrasound needle segmentation and trajectory prediction using excitation network

Cited by 36 publications

References 16 publications

Deep learning segmentation of general interventional tools in two‐dimensional ultrasound images

Deep learning segmentation of general interventional tools in two‐dimensional ultrasound images

Gabor filter-based localization of straight and curved needles in 2D ultrasound images

Learning and Reasoning with the Graph Structure Representation in Robotic Surgery

Contact Info

Product

Resources

About