Deep Complementary Joint Model for Complex Scene Registration and Few-Shot Segmentation on Medical Images

He, Yuting; Li, Tiantian; Yang, Guanyu; Kong, Youyong; Yang, Chen; Hu, Shu; Coatrieux, Jean-Louis; Dillenseger, Jean-Louis; Li, Shuo

doi:10.1007/978-3-030-58523-5_45

Cited by 19 publications

(24 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, these approaches are amenable to training with few segmented examples including semi-supervised learning. For example, a registration network was used to segment unlabeled data for training [10], [13] as well as create augmented samples through random perturbations in the warped images [9]. These methods benefit by using the segmentation network to provide additional regularization losses to optimize the registration network training such as through segmentation consistency [13] and cycle consistency losses [10].…”

Section: A Medical Image Registration-based Segmentationmentioning

confidence: 99%

“…Multi-task networks [9], [13]- [15] handle limited datasets by using implicit data augmentation available from the different tasks through the losses to jointly optimize registration and segmentation. Notably, these methods have shown feasibility for one and few-shot normal organ segmentation [9], [13], [16], using CT-to-CT or MRI-to-MRI registration. Planning CT to CBCT registration is harder because of low soft-tissue contrast and narrow field of view (FOV) on CBCT [12].…”

Section: Introductionmentioning

confidence: 99%

“…Prior works applied to CBCT registration aligned large moving organs [4], [9], [12], [17], [18]. We tackle more challenging lung tumor segmentation from CBCT for longitudinal response assessment in tumors undergoing treatment.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

One shot PACS: Patient specific Anatomic Context and Shape prior aware recurrent registration-segmentation of longitudinal thoracic cone beam CTs

Jiang,

Veeraraghavan

2022

Preprint

View full text Add to dashboard Cite

Image-guided adaptive lung radiotherapy requires accurate tumor and organs segmentation from during treatment cone-beam CT (CBCT) images. Thoracic CBCTs are hard to segment because of low soft-tissue contrast, imaging artifacts, respiratory motion, and large treatment induced intra-thoracic anatomic changes. Hence, we developed a novel Patient-specific Anatomic Context and Shape prior or PACS-aware 3D recurrent registrationsegmentation network for longitudinal thoracic CBCT segmentation. Segmentation and registration networks were concurrently trained in an end-to-end framework and implemented with convolutional long-short term memory models. The registration network was trained in an unsupervised manner using pairs of planning CT (pCT) and CBCT images and produced a progressively deformed sequence of images. The segmentation network was optimized in a one-shot setting by combining progressively deformed pCT (anatomic context) and pCT delineations (shape context) with CBCT images. Our method, one-shot PACS was significantly more accurate (p <0.001) for tumor (DSC of 0.83 ± 0.08, surface DSC [sDSC] of 0.97 ± 0.06, and Hausdorff distance at 95 th percentile [HD95] of 3.97±3.02mm) and the esophagus (DSC of 0.78 ± 0.13, sDSC of 0.90±0.14, HD95 of 3.22±2.02) segmentation than multiple methods. Ablation tests and comparative experiments were also done.

show abstract

Section: A Medical Image Registration-based Segmentationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

One shot PACS: Patient specific Anatomic Context and Shape prior aware recurrent registration-segmentation of longitudinal thoracic cone beam CTs

Jiang,

Veeraraghavan

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Recently, deep learning (DL)-based methods [5,10,19] have gained increasing popularity in the field of deformable image registration. DL-based methods [2,16,26,28,33] learn a universal representations of data samples using convolutional neural networks (CNN) to estimate corresponding deformation between the moving and fixed images.…”

Section: Introductionmentioning

confidence: 99%

“…In addition, these methods only use a global similarity metric to evaluate the similarity of a pair of images, which cannot maximize the similarity of ROIs within a pair of images. Furthermore, label-constrained (LC) registration methods [13,19,20,27] have been developed to enhance the correspondence of ROIs between a pair of images. LC registration approaches [5,18,30,42] are capable of leveraging auxiliary information such as segmentation labels, and thus the appearance structure of ROIs can be perceived, which further enhances the similarity of ROIs within images.…”

Section: Introductionmentioning

confidence: 99%

Dual-Flow Transformation Network for Deformable Image Registration with Region Consistency Constraint

Ma¹,

Yibo²,

Xia³

et al. 2021

Preprint

View full text Add to dashboard Cite

Deformable image registration is able to achieve fast and accurate alignment between a pair of images and thus plays an important role in many medical image studies. The current deep learning (DL)-based image registration approaches directly learn the spatial transformation from one image to another by leveraging a convolutional neural network, requiring ground truth or similarity metric. Nevertheless, these methods only use a global similarity energy function to evaluate the similarity of a pair of images, which ignores the similarity of regions of interest (ROIs) within images. Moreover, DL-based methods often estimate global spatial transformations of image directly, which never pays attention to region spatial transformations of ROIs within images. In this paper, we present a novel dual-flow transformation network with region consistency constraint which maximizes the similarity of ROIs within a pair of images and estimates both global and region spatial transformations simultaneously. Experiments on four public 3D MRI datasets show that the proposed method achieves the best registration performance in accuracy and generalization compared with other state-of-the-art methods.

show abstract

Progressively refined deep joint registration segmentation (ProRSeg) of gastrointestinal organs at risk: Application to MRI and cone‐beam CT

et al. 2023

View full text Add to dashboard Cite

BackgroundAdaptive radiation treatment (ART) for locally advanced pancreatic cancer (LAPC) requires consistently accurate segmentation of the extremely mobile gastrointestinal (GI) organs at risk (OAR) including the stomach, duodenum, large and small bowel. Also, due to lack of sufficiently accurate and fast deformable image registration (DIR), accumulated dose to the GI OARs is currently only approximated, further limiting the ability to more precisely adapt treatments.PurposeDevelop a 3‐D Progressively refined joint Registration‐Segmentation (ProRSeg) deep network to deformably align and segment treatment fraction magnetic resonance images (MRI)s, then evaluate segmentation accuracy, registration consistency, and feasibility for OAR dose accumulation.MethodProRSeg was trained using five‐fold cross‐validation with 110 T2‐weighted MRI acquired at five treatment fractions from 10 different patients, taking care that same patient scans were not placed in training and testing folds. Segmentation accuracy was measured using Dice similarity coefficient (DSC) and Hausdorff distance at 95th percentile (HD95). Registration consistency was measured using coefficient of variation (CV) in displacement of OARs. Statistical comparison to other deep learning and iterative registration methods were done using the Kruskal‐Wallis test, followed by pair‐wise comparisons with Bonferroni correction applied for multiple testing. Ablation tests and accuracy comparisons against multiple methods were done. Finally, applicability of ProRSeg to segment cone‐beam CT (CBCT) scans was evaluated on a publicly available dataset of 80 scans using five‐fold cross‐validation.ResultsProRSeg processed 3D volumes (128 × 192 × 128) in 3 s on a NVIDIA Tesla V100 GPU. It's segmentations were significantly more accurate () than compared methods, achieving a DSC of 0.94 ±0.02 for liver, 0.88±0.04 for large bowel, 0.78±0.03 for small bowel and 0.82±0.04 for stomach‐duodenum from MRI. ProRSeg achieved a DSC of 0.72±0.01 for small bowel and 0.76±0.03 for stomach‐duodenum from public CBCT dataset. ProRSeg registrations resulted in the lowest CV in displacement (stomach‐duodenum : 0.75%, : 0.73%, and : 0.81%; small bowel : 0.80%, : 0.80%, and : 0.68%; large bowel : 0.71%, : 0.81%, and : 0.75%). ProRSeg based dose accumulation accounting for intra‐fraction (pre‐treatment to post‐treatment MRI scan) and inter‐fraction motion showed that the organ dose constraints were violated in four patients for stomach‐duodenum and for three patients for small bowel. Study limitations include lack of independent testing and ground truth phantom datasets to measure dose accumulation accuracy.ConclusionsProRSeg produced more accurate and consistent GI OARs segmentation and DIR of MRI and CBCTs compared to multiple methods. Preliminary results indicates feasibility for OAR dose accumulation using ProRSeg.

show abstract

Deep Complementary Joint Model for Complex Scene Registration and Few-Shot Segmentation on Medical Images

Cited by 19 publications

References 34 publications

One shot PACS: Patient specific Anatomic Context and Shape prior aware recurrent registration-segmentation of longitudinal thoracic cone beam CTs

One shot PACS: Patient specific Anatomic Context and Shape prior aware recurrent registration-segmentation of longitudinal thoracic cone beam CTs

Dual-Flow Transformation Network for Deformable Image Registration with Region Consistency Constraint

Progressively refined deep joint registration segmentation (ProRSeg) of gastrointestinal organs at risk: Application to MRI and cone‐beam CT

Contact Info

Product

Resources

About