Transformer based Generative Adversarial Network for Liver Segmentation

Demir, Uğur; Zhang, Zheyuan; Wang, Bin; Antalek, Matthew; Keleş, Elif; Jha, Debesh; Borhani, Amir A.; Ladner, Daniela P.; Bağcı, Ulaş

doi:10.48550/arxiv.2205.10663

Cited by 2 publications

(2 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Over the past few years, there has been a dramatic increase in the use of convolutional neural networks (CNN) in computer vision and medical imaging applications; particular attention is focused on the U-Net style segmentors and, more recently, combined with Transformers ( 9 – 13 ). Here, we briefly review the mostly used segmentation architectures and their characteristics, and choose common baselines for our current study.…”

Section: Introductionmentioning

confidence: 99%

Selecting the best optimizers for deep learning–based medical image segmentation

Mortazi,

Cicek,

Keles

et al. 2023

Front. Radiol.

View full text Add to dashboard Cite

PurposeThe goal of this work is to explore the best optimizers for deep learning in the context of medical image segmentation and to provide guidance on how to design segmentation networks with effective optimization strategies.ApproachMost successful deep learning networks are trained using two types of stochastic gradient descent (SGD) algorithms: adaptive learning and accelerated schemes. Adaptive learning helps with fast convergence by starting with a larger learning rate (LR) and gradually decreasing it. Momentum optimizers are particularly effective at quickly optimizing neural networks within the accelerated schemes category. By revealing the potential interplay between these two types of algorithms [LR and momentum optimizers or momentum rate (MR) in short], in this article, we explore the two variants of SGD algorithms in a single setting. We suggest using cyclic learning as the base optimizer and integrating optimal values of learning rate and momentum rate. The new optimization function proposed in this work is based on the Nesterov accelerated gradient optimizer, which is more efficient computationally and has better generalization capabilities compared to other adaptive optimizers.ResultsWe investigated the relationship of LR and MR under an important problem of medical image segmentation of cardiac structures from MRI and CT scans. We conducted experiments using the cardiac imaging dataset from the ACDC challenge of MICCAI 2017, and four different architectures were shown to be successful for cardiac image segmentation problems. Our comprehensive evaluations demonstrated that the proposed optimizer achieved better results (over a 2% improvement in the dice metric) than other optimizers in the deep learning literature with similar or lower computational cost in both single and multi-object segmentation settings.ConclusionsWe hypothesized that the combination of accelerated and adaptive optimization methods can have a drastic effect in medical image segmentation performances. To this end, we proposed a new cyclic optimization method (Cyclic Learning/Momentum Rate) to address the efficiency and accuracy problems in deep learning–based medical image segmentation. The proposed strategy yielded better generalization in comparison to adaptive optimizers.

show abstract

Section: Introductionmentioning

confidence: 99%

Selecting the best optimizers for deep learning–based medical image segmentation

Mortazi,

Cicek,

Keles

et al. 2023

Front. Radiol.

View full text Add to dashboard Cite

show abstract

“…[17][18][19][20][21] The successful CNN-based segmentation approaches can be divided into three broad categories. The first category is named encoder-decoder architecture.…”

mentioning

confidence: 99%

Selecting the Best Optimizers for Deep Learning based Medical Image Segmentation

Mortazi¹,

Çiçek²,

Keleş³

et al. 2023

Preprint

View full text Add to dashboard Cite

Purpose: The goal of this work is to identify the best optimizers for deep learning in the context of cardiac image segmentation and to provide guidance on how to design segmentation networks with effective optimization strategies. Approach: Most successful deep learning networks are trained using two types of stochastic gradient descent (SGD) algorithms: adaptive learning and accelerated schemes. Adaptive learning helps with fast convergence by starting with a larger learning rate (LR) and gradually decreasing it. Momentum optimizers are particularly effective at quickly optimizing neural networks within the accelerated schemes category. By revealing the potential interplay between these two types of algorithms (LR and momentum optimizers or momentum rate (MR) in short), in this article, we explore the two variants of SGD algorithms in a single setting. We suggest using cyclic learning as the base optimizer and integrating optimal values of learning rate and momentum rate. The new optimization function proposed in this work is based on the Nesterov accelerated gradient optimizer, which is more efficient computationally and has better generalization capabilities compared to other adaptive optimizers. Results: We investigated the relationship of LR and MR under an important problem of medical image segmentation of cardiac structures from MRI and CT scans. We conducted experiments using the cardiac imaging dataset from the ACDC challenge of MICCAI 2017, and four different architectures shown to be successful for cardiac image segmentation problems. Our comprehensive evaluations demonstrated that the proposed optimizer achieved better results (over a 2% improvement in the dice metric) than other optimizers in deep learning literature with similar or lower computational cost in both single and multi-object segmentation settings. Conclusions: We hypothesized that combination of accelerated and adaptive optimization methods can have a drastic effect in medical image segmentation performances. To this end, we proposed a new cyclic optimization method (CLMR) to address the efficiency and accuracy problems in deep learning based medical image segmentation. The proposed strategy yielded better generalization in comparison to adaptive optimizers.

show abstract

Transformer based Generative Adversarial Network for Liver Segmentation

Cited by 2 publications

References 11 publications

Selecting the best optimizers for deep learning–based medical image segmentation

Selecting the best optimizers for deep learning–based medical image segmentation

Selecting the Best Optimizers for Deep Learning based Medical Image Segmentation

Contact Info

Product

Resources

About