Integrating multiple MRI sequences for pelvic organs segmentation via the attention mechanism

Abstract

Purpose To create a network which fully utilizes multi‐sequence MRI and compares favorably with manual human contouring. Methods We retrospectively collected 89 MRI studies of the pelvic cavity from patients with prostate cancer and cervical cancer. The dataset contained 89 samples from 87 patients with a total of 84 valid samples. MRI was performed with T1‐weighted (T1), T2‐weighted (T2), and Enhanced Dixon T1‐weighted (T1DIXONC) sequences. There were two cohorts. The training cohort contained 55 samples and the testing cohort contained 29 samples. The MRI images in the training cohort contained contouring data from radiotherapist α. The MRI images in the testing cohort contained contouring data from radiotherapist α and contouring data from another radiotherapist: radiotherapist β. The training cohort was used to optimize the convolution neural networks, which included the attention mechanism through the proposed activation module and the blended module into multiple MRI sequences, to perform autodelineation. The testing cohort was used to assess the networks’ autodelineation performance. The contoured organs at risk (OAR) were the anal canal, bladder, rectum, femoral head (L), and femoral head (R). Results We compared our proposed network with UNet and FuseUNet using our dataset. When T1 was the main sequence, we input three sequences to segment five organs and evaluated the results using four metrics: the DSC (Dice similarity coefficient), the JSC (Jaccard similarity coefficient), the ASD (average mean distance), and the 95% HD (robust Hausdorff distance). The proposed network achieved improved results compared with the baselines among all metrics. The DSC were 0.834±0.029, 0.818±0.037, and 0.808±0.050 for our proposed network, FuseUNet, and UNet, respectively. The 95% HD were 7.256±2.748 mm, 8.404±3.297 mm, and 8.951±4.798 mm for our proposed network, FuseUNet, and UNet, respectively. Our proposed network also had superior performance on the JSC and ASD coefficients. Conclusion Our proposed activation module and blended module significantly improved the performance of FuseUNet for multi‐sequence MRI segmentation. Our proposed network integrated multiple MRI sequences efficiently and autosegmented OAR rapidly and accurately. We also discovered that three‐sequence fusion (T1‐T1DIXONC‐T2) was superior to two‐sequence fusion (T1‐T2 and T1‐T1DIXONC, respectively). We infer that the more MRI sequences fused, the better the automatic segmentation results.

Prior knowledge based deep learning auto-segmentation in magnetic resonance imaging-guided radiotherapy of prostate cancer

“…For OARs, the PSMs gave the best outcomes in terms of DSC and HDs, comparable to the state-of-the art in automatic pelvic segmentation [17] , [30] , [31] , [32] . For the bladder, PSMs mostly corrected larger volume misclassifications of the BM as observed in patient 01.…”

Fully automated bladder tumor segmentation from T2 MRI images using 3D U-Net algorithm

“…Before mpMRI, contrast-enhanced computer tomography was used for bladder tumors segmentation. Although this imaging modality reaches a good accuracy in terms of automatic tumor detection (84.2% on a study database of 182), in terms of preoperative grade assessment, the accuracy drops to 77.9% ( 10 ), this being mainly attributed to the limited contrast between adjacent soft tissue structures ( 11 ).…”

Cervical cancer segmentation based on medical images: a literature review

“…To address this, researchers have explored the use of multiple MRI sequences to improve segmentation accuracy. Huang et al ( 70 ) proposed a modified FuseNet network to segment the organs at risk of cervical cancer and prostate cancer. The network used an attention mechanism to fuse multimodal images from T1-weighted, T2-weighted, and enhanced Dixon T1-weighted sequences.…”