Semi Supervised Semantic Segmentation Using Generative Adversarial Network

Souly, Nasim; Spampinato, Concetto; Shah, Mubarak

doi:10.1109/iccv.2017.606

Cited by 472 publications

(280 citation statements)

References 29 publications

Supporting

Mentioning

280

Contrasting

Order By: Relevance

“…Each label of the segmentation model represents a referred region. The segmentation model is implemented by a 2.5D end‐to‐end patch‐based GAN model, which takes four continuous slices of CT images as an input patch, that is, patch size of 512 × 512 × 4, and outputs the equal‐sized heart, left lung, and right lung segmentations. Esophagus and spinal cord segmentation are trained separately with 3D GAN on cropped region of interest (ROI) patches.…”

Section: Methodsmentioning

confidence: 99%

Automatic multiorgan segmentation in thorax CT images using U‐net‐GAN

et al. 2019

View full text Add to dashboard Cite

Purpose Accurate and timely organs‐at‐risk (OARs) segmentation is key to efficient and high‐quality radiation therapy planning. The purpose of this work is to develop a deep learning‐based method to automatically segment multiple thoracic OARs on chest computed tomography (CT) for radiotherapy treatment planning. Methods We propose an adversarial training strategy to train deep neural networks for the segmentation of multiple organs on thoracic CT images. The proposed design of adversarial networks, called U‐Net‐generative adversarial network (U‐Net‐GAN), jointly trains a set of U‐Nets as generators and fully convolutional networks (FCNs) as discriminators. Specifically, the generator, composed of U‐Net, produces an image segmentation map of multiple organs by an end‐to‐end mapping learned from CT image to multiorgan‐segmented OARs. The discriminator, structured as an FCN, discriminates between the ground truth and segmented OARs produced by the generator. The generator and discriminator compete against each other in an adversarial learning process to produce the optimal segmentation map of multiple organs. Our segmentation results were compared with manually segmented OARs (ground truth) for quantitative evaluations in geometric difference, as well as dosimetric performance by investigating the dose‐volume histogram in 20 stereotactic body radiation therapy (SBRT) lung plans. Results This segmentation technique was applied to delineate the left and right lungs, spinal cord, esophagus, and heart using 35 patients’ chest CTs. The averaged dice similarity coefficient for the above five OARs are 0.97, 0.97, 0.90, 0.75, and 0.87, respectively. The mean surface distance of the five OARs obtained with proposed method ranges between 0.4 and 1.5 mm on average among all 35 patients. The mean dose differences on the 20 SBRT lung plans are −0.001 to 0.155 Gy for the five OARs. Conclusion We have investigated a novel deep learning‐based approach with a GAN strategy to segment multiple OARs in the thorax using chest CT images and demonstrated its feasibility and reliability. This is a potentially valuable method for improving the efficiency of chest radiotherapy treatment planning.

show abstract

Section: Methodsmentioning

confidence: 99%

Automatic multiorgan segmentation in thorax CT images using U‐net‐GAN

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Nevertheless, the focus of this work was not to synthesize MR images that are identical to true MRI as is the focus of works using generational adversarial networks to synthesize CT from MR. 40 Instead our goal was to augment the ability of deep learning to learn despite limited training sets by leveraging datasets mimicking intensity variations as the real MR images. Our approach improves on the prior GAN-based data augmentation approaches, 41,42 by explicitly modeling the appearance of structures of interest, namely the tumors for generating fully automatic and longitudinal segmentation of lung tumors from MRI with access to only a few expert-segmented MRI datasets.…”

Section: Discussionmentioning

confidence: 99%

Cross‐modality (CT‐MRI) prior augmented deep learning for robust lung tumor segmentation from small MR datasets

Jiang

Tyagi

et al. 2019

Medical Physics

View full text Add to dashboard Cite

Purpose Accurate tumor segmentation is a requirement for magnetic resonance (MR)‐based radiotherapy. Lack of large expert annotated MR datasets makes training deep learning models difficult. Therefore, a cross‐modality (MR‐CT) deep learning segmentation approach that augments training data using pseudo MR images produced by transforming expert‐segmented CT images was developed. Methods Eighty‐one T2‐weighted MRI scans from 28 patients with non‐small cell lung cancers (nine with pretreatment and weekly MRI and the remainder with pre‐treatment MRI scans) were analyzed. Cross‐modality model encoding the transformation of CT to pseudo MR images resembling T2w MRI was learned as a generative adversarial deep learning network. This model was used to translate 377 expert segmented non‐small cell lung cancer CT scans from the Cancer Imaging Archive into pseudo MRI that served as additional training set. This method was benchmarked against shallow learning using random forest, standard data augmentation, and three state‐of‐the art adversarial learning‐based cross‐modality data (pseudo MR) augmentation methods. Segmentation accuracy was computed using Dice similarity coefficient (DSC), Hausdorff distance metrics, and volume ratio. Results The proposed approach produced the lowest statistical variability in the intensity distribution between pseudo and T2w MR images measured as Kullback–Leibler divergence of 0.069. This method produced the highest segmentation accuracy with a DSC of (0.75 ± 0.12) and the lowest Hausdorff distance of (9.36 mm ± 6.00 mm) on the test dataset using a U‐Net structure. This approach produced highly similar estimations of tumor growth as an expert (P = 0.37). Conclusions A novel deep learning MR segmentation was developed that overcomes the limitation of learning robust models from small datasets by leveraging learned cross‐modality information using a model that explicitly incorporates knowledge of tumors in modality translation to augment segmentation training. The results show the feasibility of the approach and the corresponding improvement over the state‐of‐the‐art methods.

show abstract

“…Numerous works such as [9,10,11,12] have further extended and improved the original vanilla GAN [8]. Moreover, it has been used in a wide variety of applications including image generation [9,13], domain adaptation [14,15,16,17], object detection [18,19], video applications [20,21,22] and semantic segmentation [23,24,25,26].…”

Section: Generative Adversarial Networkmentioning

confidence: 99%

Object Segmentation Using Pixel-Wise Adversarial Loss

Durall

Pfreundt

Köthe³

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Recent deep learning based approaches have shown remarkable success on object segmentation tasks. However, there is still room for further improvement. Inspired by generative adversarial networks, we present a generic end-to-end adversarial approach, which can be combined with a wide range of existing semantic segmentation networks to improve their segmentation performance. The key element of our method is to replace the commonly used binary adversarial loss with a high resolution pixel-wise loss. In addition, we train our generator employing stochastic weight averaging fashion, which further enhances the predicted output label maps leading to state-of-the-art results. We show, that this combination of pixel-wise adversarial training and weight averaging leads to significant and consistent gains in segmentation performance, compared to the baseline models.

show abstract

Semi Supervised Semantic Segmentation Using Generative Adversarial Network

Cited by 472 publications

References 29 publications

Automatic multiorgan segmentation in thorax CT images using U‐net‐GAN

Automatic multiorgan segmentation in thorax CT images using U‐net‐GAN

Cross‐modality (CT‐MRI) prior augmented deep learning for robust lung tumor segmentation from small MR datasets

Object Segmentation Using Pixel-Wise Adversarial Loss

Contact Info

Product

Resources

About