Deep learning‐based digitization of prostate brachytherapy needles in ultrasound images

Abstract

To develop, and evaluate the performance of, a deep learning-based three-dimensional (3D) convolutional neural network (CNN) artificial intelligence (AI) algorithm aimed at finding needles in ultrasound images used in prostate brachytherapy. Methods: Transrectal ultrasound (TRUS) image volumes from 1102 treatments were used to create a clinical ground truth (CGT) including 24422 individual needles that had been manually digitized by medical physicists during brachytherapy procedures. A 3D CNN U-net with 128 × 128 × 128 TRUS image volumes as input was trained using 17215 needle examples. Predictions of voxels constituting a needle were combined to yield a 3D linear function describing the localization of each needle in a TRUS volume. Manual and AI digitizations were compared in terms of the root-mean-square distance (RMSD) along each needle, expressed as median and interquartile range (IQR). The method was evaluated on a data set including 7207 needle examples. A subgroup of the evaluation data set (n = 188) was created, where the needles were digitized once more by a medical physicist (G1) trained in brachytherapy. The digitization procedure was timed. Results: The RMSD between the AI and CGT was 0.55 (IQR: 0.35-0.86) mm. In the smaller subset, the RMSD between AI and CGT was similar (0.52 [IQR: 0.33-0.79] mm) but significantly smaller (P < 0.001) than the difference of 0.75 (IQR: 0.49-1.20) mm between AI and G1. The difference between CGT and G1 was 0.80 (IQR: 0.48-1.18) mm, implying that the AI performed as well as the CGT in relation to G1. The mean time needed for human digitization was 10 min 11 sec, while the time needed for the AI was negligible. Conclusions: A 3D CNN can be trained to identify needles in TRUS images. The performance of the network was similar to that of a medical physicist trained in brachytherapy. Incorporating a CNN for needle identification can shorten brachytherapy treatment procedures substantially.

Observer preference of artificial intelligence‐generated versus clinical prostate contours for ultrasound‐based high dose rate brachytherapy

“…This algorithm can then be tested prospectively, ideally with less-experienced and more-experienced observers across multiple centers, to evaluate whether such an AI algorithm can reduce errors in prostate delineation. This algorithm could also be utilized in conjunction with deep-learning algorithms for catheter digitization on TRUS, 17,18 which may allow for more accurate and rapid treatment planning.…”

Deep‐learning‐assisted algorithm for catheter reconstruction during MR‐only gynecological interstitial brachytherapy

“…In our approach, segmentation regions were predicted for the entire image volume by making predictions from previous slices. This work is partially dependent on the original form of the UNet deep learning model developed by Ronneberger et al 21 This model was chosen because (i) this model is based on the convolution neural network (CNN), which has been extensively used to develop automated accurate and stable detection and segmentation methods for the clinical target volume and brachytherapy catheters on US, MR and CT images during prostate and gynecological brachytherapy, 18,24,25 and (ii) learned features from UNet CNN layers can be recognized regardless of their position in the image.This makes it useful for processing images with similar features (e.g., catheter positions), and is robust against variations in feature position or imaging conditions. 26 Few studies have investigated the use of fully automatic or semiautomatic reconstruction of interstitial catheters during gynecological HDR brachytherapy.…”

Medical instrument detection in ultrasound: a review

“…However, for 3D volumetric data, the decomposition approach may limit semantic information usage due to the compromised 3D information after slicing. To address this, patch-based 2.5D or 3D U-Net were proposed to segment the cardiac catheter (Yang et al 2019a(Yang et al , c, f, 2020a or prostate needles (Zhang et al 2020d;Andersén et al 2020) in 3D volumetric data by dividing the image into smaller patches or reducing the image size. In this way, the 3D contextual information is preserved and the requirement on GPU memory is reduced for 3D deep learning.…”