Momentum Contrastive Voxel-Wise Representation Learning for Semi-supervised Volumetric Medical Image Segmentation

You, Chenyu; Zhao, Ruihan; Staib, Lawrence H.; Duncan, James S.

doi:10.1007/978-3-031-16440-8_61

Cited by 64 publications

(14 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep learning models have proven to be quite effective at identifying important and distinct characteristics, particularly in image data 27,28 . Deep models have the capability to outline regions of interest automatically, capture textural changes within a lesion, discriminate between cancerous and non‐cancerous cells, and potentially extract distinctive information from lesions to be later used for the task of outcome prediction 29–38 . Diamant et al 39 .…”

Section: Introductionmentioning

confidence: 99%

Predicting the outcome of radiotherapy in brain metastasis by integrating the clinical and MRI‐based deep learning features

et al. 2022

View full text Add to dashboard Cite

Background A considerable proportion of metastatic brain tumors progress locally despite stereotactic radiation treatment, and it can take months before such local progression is evident on follow‐up imaging. Prediction of radiotherapy outcome in terms of tumor local failure is crucial for these patients and can facilitate treatment adjustments or allow for early salvage therapies. Purpose In this work, a novel deep learning architecture is introduced to predict the outcome of local control/failure in brain metastasis treated with stereotactic radiation therapy using treatment‐planning magnetic resonance imaging (MRI) and standard clinical attributes. Methods At the core of the proposed architecture is an InceptionResentV2 network to extract distinct features from each MRI slice for local outcome prediction. A recurrent or transformer network is integrated into the architecture to incorporate spatial dependencies between MRI slices into the predictive modeling. A visualization method based on prediction difference analysis is coupled with the deep learning model to illustrate how different regions of each lesion on MRI contribute to the model's prediction. The model was trained and optimized using the data acquired from 99 patients (116 lesions) and evaluated on an independent test set of 25 patients (40 lesions). Results The results demonstrate the promising potential of the MRI deep learning features for outcome prediction, outperforming standard clinical variables. The prediction model with only clinical variables demonstrated an area under the receiver operating characteristic curve (AUC) of 0.68. The MRI deep learning models resulted in AUCs in the range of 0.72 to 0.83 depending on the mechanism to integrate information from MRI slices of each lesion. The best prediction performance (AUC = 0.86) was associated with the model that combined the MRI deep learning features with clinical variables and incorporated the inter‐slice dependencies using a long short‐term memory recurrent network. The visualization results highlighted the importance of tumor/lesion margins in local outcome prediction for brain metastasis. Conclusions The promising results of this study show the possibility of early prediction of radiotherapy outcome for brain metastasis via deep learning of MRI and clinical attributes at pre‐treatment and encourage future studies on larger groups of patients treated with other radiotherapy modalities.

show abstract

Section: Introductionmentioning

confidence: 99%

Predicting the outcome of radiotherapy in brain metastasis by integrating the clinical and MRI‐based deep learning features

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Swin-Unet introduces a pure U-shaped transformer using Swin transformer (Liu et al, 2021) blocks. Recently, in CASTFormer (You et al, 2022), authors introduce a class-aware transformer with adversarial training.…”

Section: Medical Image Segmentationmentioning

confidence: 99%

Medical Image Segmentation via Cascaded Attention Decoding

Rahman

Mărculescu

2023

2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

100

View full text Add to dashboard Cite

Transformers have shown great success in medical image segmentation. However, transformers may exhibit a limited generalization ability due to the underlying single-scale selfattention (SA) mechanism. In this paper, we address this issue by introducing a Multiscale hiERarchical vIsion Transformer (MERIT) backbone network, which improves the generalizability of the model by computing SA at multiple scales. We also incorporate an attention-based decoder, namely Cascaded Attention Decoding (CASCADE), for further refinement of multi-stage features generated by MERIT. Finally, we introduce an effective multi-stage feature mixing loss aggregation (MUTATION) method for better model training via implicit ensembling. Our experiments on two widely used medical image segmentation benchmarks (i.e., Synapse Multi-organ, ACDC) demonstrate the superior performance of MERIT over state-of-the-art methods. Our MERIT architecture and MUTATION loss aggregation can be used with downstream medical image and semantic segmentation tasks.

show abstract

“…With the aim of improving image reconstruction and ease the learning process of anatomical features, in Ref. [90], the authors proposed a contrastive voxel‐wise representation learning in CT images and, in Ref. [91], presented and validated an improved methodology named simple contrastive voxel‐wise representation distillation (SimCVD) for CT image segmentation, which significantly improved SOA voxel‐wise representation learning.…”

Section: The “Ideal” Probe Design Considerationsmentioning

confidence: 99%

Nuclear‐medicine probes: Where we are and where we are going

González-Montoro

Donoso²,

Konstantinou³

et al. 2022

Medical Physics

View full text Add to dashboard Cite

Nuclear medicine probes turned into the key for the identification and precise location of sentinel lymph nodes and other occult lesions (i.e., tumors) by using the systemic administration of radiotracers. Intraoperative nuclear probes are key in the surgical management of some malignancies as well as in the determination of positive surgical margins, thus reducing the extent and potential surgery morbidity. Depending on their application, nuclear probes are classified into two main categories, namely, counting and imaging. Although counting probes present a simple design, are handheld (to be moved rapidly), and provide only acoustic signals when detecting radiation, imaging probes, also known as cameras, are more hardware‐complex and also able to provide images but at the cost of an increased intervention time as displacing the camera has to be done slowly. This review article begins with an introductory section to highlight the relevance of nuclear‐based probes and their components as well as the main differences between ionization‐ (semiconductor) and scintillation‐based probes. Then, the most significant performance parameters of the probe are reviewed (i.e., sensitivity, contrast, count rate capabilities, shielding, energy, and spatial resolution), as well as the different types of probes based on the target radiation nature, namely: gamma (γ), beta (β) (positron and electron), and Cherenkov. Various available intraoperative nuclear probes are finally compared in terms of performance to discuss the state‐of‐the‐art of nuclear medicine probes. The manuscript concludes by discussing the ideal probe design and the aspects to be considered when selecting nuclear‐medicine probes.

show abstract

Momentum Contrastive Voxel-Wise Representation Learning for Semi-supervised Volumetric Medical Image Segmentation

Cited by 64 publications

References 27 publications

Predicting the outcome of radiotherapy in brain metastasis by integrating the clinical and MRI‐based deep learning features

Predicting the outcome of radiotherapy in brain metastasis by integrating the clinical and MRI‐based deep learning features

Medical Image Segmentation via Cascaded Attention Decoding

Nuclear‐medicine probes: Where we are and where we are going

Contact Info

Product

Resources

About