A deep learning framework for automatic detection of arbitrarily shaped fiducial markers in intrafraction fluoroscopic images

Abstract

Purpose Real‐time image‐guided adaptive radiation therapy (IGART) requires accurate marker segmentation to resolve three‐dimensional (3D) motion based on two‐dimensional (2D) fluoroscopic images. Most common marker segmentation methods require prior knowledge of marker properties to construct a template. If marker properties are not known, an additional learning period is required to build the template which exposes the patient to an additional imaging dose. This work investigates a deep learning‐based fiducial marker classifier for use in real‐time IGART that requires no prior patient‐specific data or additional learning periods. The proposed tracking system uses convolutional neural network (CNN) models to segment cylindrical and arbitrarily shaped fiducial markers. Methods The tracking system uses a tracking window approach to perform sliding window classification of each implanted marker. Three cylindrical marker training datasets were generated from phantom kilovoltage (kV) and patient intrafraction images with increasing levels of megavoltage (MV) scatter. The cylindrical shaped marker CNNs were validated on unseen kV fluoroscopic images from 12 fractions of 10 prostate cancer patients with implanted gold fiducials. For the training and validation of the arbitrarily shaped marker CNNs, cone beam computed tomography (CBCT) projection images from ten fractions of seven lung cancer patients with implanted coiled markers were used. The arbitrarily shaped marker CNNs were trained using three patients and the other four unseen patients were used for validation. The effects of full training using a compact CNN (four layers with learnable weights) and transfer learning using a pretrained CNN (AlexNet, eight layers with learnable weights) were analyzed. Each CNN was evaluated using a Precision‐Recall curve (PRC), the area under the PRC plot (AUC), and by the calculation of sensitivity and specificity. The tracking system was assessed using the validation data and the accuracy was quantified by calculating the mean error, root‐mean‐square error (RMSE) and the 1st and 99th percentiles of the error. Results The fully trained CNN on the dataset with moderate noise levels had a sensitivity of 99.00% and specificity of 98.92%. Transfer learning of AlexNet resulted in a sensitivity and specificity of 99.42% and 98.13%, respectively, for the same datasets. For the arbitrarily shaped marker CNNs, the sensitivity was 98.58% and specificity was 98.97% for the fully trained CNN. The transfer learning CNN had a sensitivity and specificity of 98.49% and 99.56%, respectively. The CNNs were successfully incorporated into a multiple object tracking system for both cylindrical and arbitrarily shaped markers. The cylindrical shaped marker tracking had a mean RMSE of 1.6 ± 0.2 pixels and 1.3 ± 0.4 pixels in the x‐ and y‐directions, respectively. The arbitrarily shaped marker tracking had a mean RMSE of 3.0 ± 0.5 pixels and 2.2 ± 0.4 pixels in the x‐ and y‐directions, respectively. Conclusion With deep learning CNNs, high classification...

The accuracy and precision of the KIM motion monitoring system used in the multi‐institutional TROG 15.01 Stereotactic Prostate Ablative Radiotherapy with KIM (SPARK) trial

“…A novel Wiener‐based filter has been investigated for KIM that has been found to improve image quality . A deep learning framework that classifies fiducial markers using a convolutional neural network could potentially improve on template matching with the added benefit of eliminating the need for prior knowledge of the marker properties.…”

The role of artificial intelligence in medical imaging research

“…Recent proposed deep learning CNN framework requires no prior knowledge of marker properties or additional learning periods to segment cylindrical and arbitrarily shaped fiducial markers. 22 The algorithm achieved high classification performance.…”

Machine learning applications in radiation oncology

“…Implanted fiducial markers, visualised on planar kilovoltage imaging, can act as a surrogate for tumour location in the prostate, liver and pancreas. Mylonas et al used CNNs to improve detection of these fiducials during treatment; the advantages over typical template matching approaches being the ability to detect arbitrarily shaped markers with limited prior information [147].…”