CT‐based multi‐organ segmentation using a 3D self‐attention U‐net network for pancreatic radiotherapy

Liu, Yingzi; Lei, Yang; Fu, Yabo; Wang, Tonghe; Tang, Xiangyang; Jiang, Xiaojun; Curran, Walter J.; Liu, Tian; Patel, Pretesh; Yang, Xiaofeng

doi:10.1002/mp.14386

Cited by 41 publications

(43 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…49 Recent segmentation studies start using HD95 that measures 95% quantile of distance. [50][51][52][53][54] Its value quantifies the large segmentation error and are closer to the intuition on the contour discrepancy. Futures studies from other groups that include HD95 results might help for justification.…”

Section: Discussionsupporting

confidence: 53%

See 1 more Smart Citation

Breast tumor segmentation in 3D automatic breast ultrasound using Mask scoring R‐CNN

Lei

He²,

Yao

et al. 2020

Medical Physics

Self Cite

View full text Add to dashboard Cite

Purpose Automatic breast ultrasound (ABUS) imaging has become an essential tool in breast cancer diagnosis since it provides complementary information to other imaging modalities. Lesion segmentation on ABUS is a prerequisite step of breast cancer computer‐aided diagnosis (CAD). This work aims to develop a deep learning‐based method for breast tumor segmentation using three‐dimensional (3D) ABUS automatically. Methods For breast tumor segmentation in ABUS, we developed a Mask scoring region‐based convolutional neural network (R‐CNN) that consists of five subnetworks, that is, a backbone, a regional proposal network, a region convolutional neural network head, a mask head, and a mask score head. A network block building direct correlation between mask quality and region class was integrated into a Mask scoring R‐CNN based framework for the segmentation of new ABUS images with ambiguous regions of interest (ROIs). For segmentation accuracy evaluation, we retrospectively investigated 70 patients with breast tumor confirmed with needle biopsy and manually delineated on ABUS, of which 40 were used for fivefold cross‐validation and 30 were used for hold‐out test. The comparison between the automatic breast tumor segmentations and the manual contours was quantified by I) six metrics including Dice similarity coefficient (DSC), Jaccard index, 95% Hausdorff distance (HD95), mean surface distance (MSD), residual mean square distance (RMSD), and center of mass distance (CMD); II) Pearson correlation analysis and Bland–Altman analysis. Results The mean (median) DSC was 85% ± 10.4% (89.4%) and 82.1% ± 14.5% (85.6%) for cross‐validation and hold‐out test, respectively. The corresponding HD95, MSD, RMSD, and CMD of the two tests was 1.646 ± 1.191 and 1.665 ± 1.129 mm, 0.489 ± 0.406 and 0.475 ± 0.371 mm, 0.755 ± 0.755 and 0.751 ± 0.508 mm, and 0.672 ± 0.612 and 0.665 ± 0.729 mm. The mean volumetric difference (mean and ± 1.96 standard deviation) was 0.47 cc ([−0.77, 1.71)) for the cross‐validation and 0.23 cc ([−0.23 0.69]) for hold‐out test, respectively. Conclusion We developed a novel Mask scoring R‐CNN approach for the automated segmentation of the breast tumor in ABUS images and demonstrated its accuracy for breast tumor segmentation. Our learning‐based method can potentially assist the clinical CAD of breast cancer using 3D ABUS imaging.

show abstract

Section: Discussionsupporting

confidence: 53%

“…However, HD is known to be overly sensitive to noise and outliers on the contours 49 . Recent segmentation studies start using HD95 that measures 95% quantile of distance 50–54 . Its value quantifies the large segmentation error and are closer to the intuition on the contour discrepancy.…”

Section: Discussionmentioning

confidence: 99%

Breast tumor segmentation in 3D automatic breast ultrasound using Mask scoring R‐CNN

Lei

He²,

Yao

et al. 2020

Medical Physics

Self Cite

View full text Add to dashboard Cite

show abstract

“…The superior segmentation results of the proposed method over the two competing methods can be attributed to a few factors. First, recent deep learning‐based end‐to‐end semantic segmentation methods, such as U‐Net, 21 perform multi‐organ segmentation on the whole volume or on slices. The challenge of this kind of method is that different organs often have large variations in shapes and sizes, such as bladder versus urethra, which would introduce imbalance during training and thus decrease the performance of inference segmentation.…”

Section: Discussionmentioning

confidence: 99%

Male pelvic multi‐organ segmentation on transrectal ultrasound using anchor‐free mask CNN

et al. 2021

Self Cite

View full text Add to dashboard Cite

Purpose Current prostate brachytherapy uses transrectal ultrasound images for implant guidance, where contours of the prostate and organs‐at‐risk are necessary for treatment planning and dose evaluation. This work aims to develop a deep learning‐based method for male pelvic multi‐organ segmentation on transrectal ultrasound images. Methods We developed an anchor‐free mask convolutional neural network (CNN) that consists of three subnetworks, that is, a backbone, a fully convolutional one‐state object detector (FCOS), and a mask head. The backbone extracts multi‐level and multi‐scale features from an ultrasound (US) image. The FOCS utilizes these features to detect and label (classify) the volume‐of‐interests (VOIs) of organs. In contrast to the design of a previously investigated mask regional CNN (Mask R‐CNN), the FCOS is anchor‐free, which can capture the spatial correlation of multiple organs. The mask head performs segmentation on each detected VOI, where a spatial attention strategy is integrated into the mask head to focus on informative feature elements and suppress noise. For evaluation, we retrospectively investigated 83 prostate cancer patients by fivefold cross‐validation and a hold‐out test. The prostate, bladder, rectum, and urethra were segmented and compared with manual contours using the Dice similarity coefficient (DSC), 95% Hausdorff distance (HD95), mean surface distance (MSD), center of mass distance (CMD), and volume difference (VD). Results The proposed method visually outperforms two competing methods, showing better agreement with manual contours and fewer misidentified speckles. In the cross‐validation study, the respective DSC and HD95 results were as follows for each organ: bladder 0.75 ± 0.12, 2.58 ± 0.7 mm; prostate 0.93 ± 0.03, 2.28 ± 0.64 mm; rectum 0.90 ± 0.07, 1.65 ± 0.52 mm; and urethra 0.86 ± 0.07, 1.85 ± 1.71 mm. For the hold‐out tests, the DSC and HD95 results were as follows: bladder 0.76 ± 0.13, 2.93 ± 1.29 mm; prostate 0.94 ± 0.03, 2.27 ± 0.79 mm; rectum 0.92 ± 0.03, 1.90 ± 0.28 mm; and urethra 0.85 ± 0.06, 1.81 ± 0.72 mm. Segmentation was performed in under 5 seconds. Conclusion The proposed method demonstrated fast and accurate multi‐organ segmentation performance. It can expedite the contouring step of prostate brachytherapy and potentially enable auto‐planning and auto‐evaluation.

show abstract

“…To reduce the computational time, end-to-end image segmentation methods such as U-Net have been proposed for medical image segmentation. [12][13][14] Ali et al developed a U-Net variation called a Res-U network to delineate the boundaries of the LV endocardium. 15 The Res-U network utilized a modified ResNet-50 as the encoder in the U-Net-like network design to perform this task.…”

Section: Introductionmentioning

confidence: 99%

“…This method, however, is relatively slow because the repetitive classifications need to be performed on each voxel. To reduce the computational time, end‐to‐end image segmentation methods such as U‐Net have been proposed for medical image segmentation 12–14 . Ali et al developed a U‐Net variation called a Res‐U network to delineate the boundaries of the LV endocardium 15 .…”

Section: Introductionmentioning

confidence: 99%

Echocardiographic image multi‐structure segmentation using Cardiac‐SegNet

Lei

Roper

et al. 2021

Medical Physics

Self Cite

View full text Add to dashboard Cite

Purpose: Cardiac boundary segmentation of echocardiographic images is important for cardiac function assessment and disease diagnosis. However, it is challenging to segment cardiac ventricles due to the low contrast-to-noise ratio and speckle noise of the echocardiographic images. Manual segmentation is subject to interobserver variability and is too slow for real-time image-guided interventions. We aim to develop a deep learning-based method for automated multi-structure segmentation of echocardiographic images. Methods: We developed an anchor-free mask convolutional neural network (CNN), termed Cardiac-SegNet, which consists of three subnetworks, that is, a backbone, a fully convolutional one-state object detector (FCOS) head, and a mask head. The backbone extracts multi-level and multi-scale features from endocardium image. The FOCS head utilizes these features to detect and label the region-of-interests (ROIs) of the segmentation targets. Unlike the traditional mask regional CNN (Mask R-CNN) method, the FCOS head is anchor-free and can model the spatial relationship of the targets. The mask head utilizes a spatial attention strategy, which allows the network to highlight salient features to perform segmentation on each detected ROI. For evaluation, we investigated 450 patient datasets by a five-fold cross-validation and a hold-out test. The endocardium (LV Endo ) and epicardium (LV Epi ) of the left ventricle and left atrium (LA) were segmented and compared with manual contours using the Dice similarity coefficient (DSC), Hausdorff distance (HD), mean absolute distance (MAD), and center-of-mass distance (CMD). Results: Compared to U-Net and Mask R-CNN, our method achieved higher segmentation accuracy and fewer erroneous speckles. When our method was evaluated on a separate hold-out dataset at the end diastole (ED) and the end systole (ES) phases, the average DSC were 0.952 and 0.939 at ED and ES for the LV Endo , 0.965 and 0.959 at ED and ES for the LV Epi , and 0.924 and 0.926 at ED and ES for the LA. For patients with a typical image size of 549 × 788 pixels, the proposed method can perform the segmentation within 0.5 s. Conclusion:We proposed a fast and accurate method to segment echocardiographic images using an anchor-free mask CNN.

show abstract

CT‐based multi‐organ segmentation using a 3D self‐attention U‐net network for pancreatic radiotherapy

Cited by 41 publications

References 33 publications

Breast tumor segmentation in 3D automatic breast ultrasound using Mask scoring R‐CNN

Breast tumor segmentation in 3D automatic breast ultrasound using Mask scoring R‐CNN

Male pelvic multi‐organ segmentation on transrectal ultrasound using anchor‐free mask CNN

Echocardiographic image multi‐structure segmentation using Cardiac‐SegNet

Contact Info

Product

Resources

About