First U-Net Layers Contain More Domain Specific Information Than the Last Ones

Shirokikh, Boris; Zakazov, Ivan; Chernyavskiy, Alexey; Fedulova, Irina; Belyaev, Mikhail

doi:10.1007/978-3-030-60548-3_12

Cited by 22 publications

(30 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The second most popular strategy to apply transfer learning was fine-tuning certain parameters in a pretrained CNN [ 34 , 127 , 128 , 129 , 130 , 131 , 132 , 133 , 134 , 135 , 136 , 137 , 138 , 139 , 140 , 141 , 142 , 143 , 144 , 145 , 146 ]. The remaining approaches first optimized a feature extractor (typically a CNN or a SVM), and then trained a separated model (SVMs [ 30 , 45 , 147 , 148 , 149 ], long short-term memory networks [ 150 , 151 ], clustering methods [ 148 , 152 ], random forests [ 70 , 153 ], multilayer perceptrons [ 154 ], logistic regression [ 148 ], elastic net [ 155 ], CNNs [ 156 ]).…”

Section: Resultsmentioning

confidence: 99%

“…Besides, as adapting pretrained CNNs to the target domain data requires, at least, replacing the last layer of these models, researchers have likely turn fine-tuning only this randomly-initialized layer into common practice, although we found no empirical studies that supported such practice. Four surveyed articles studied different fine-tuning strategies with CNNs pretrained on ImageNet [ 96 , 134 ] and medical images [ 129 , 130 ]. The approaches that utilized ImageNet-pretrained CNNs [ 96 , 134 ] reported that fine-tuning more layers led to higher accuracy, suggesting that the first layers of ImageNet-pretrained networks—that detect low-level image characteristics, such as corners and borders—may not be adequate for medical images.…”

Section: Discussionmentioning

confidence: 99%

“…Furthermore, Bashyam et al [ 101 ] reported that Inceptionv2 [ 161 ] pretrained on medical images outperformed its ImageNet-pretrained version in Alzheimer’s, mild cognitive impairment, and schizophrenia classification. Finally, a recent study [ 130 ] showed that fine-tuning the first layers of a CNN yielded better results than the traditional approach of exclusively fine-tuning the last layers, suggesting that the first layers encapsulate more domain-dependent information.…”

Section: Discussionmentioning

confidence: 99%

“…Likewise, the training set size in the target domain could be task-dependent. Only a few studies investigated the application of transfer learning with different training set sizes [ 99 , 101 , 128 , 129 , 130 , 135 , 147 , 150 ]. Among these, most articles reported that with a sufficiently large training set, models trained from scratch achieved similar or even better [ 99 , 135 ] results than applying transfer learning.…”

Section: Discussionmentioning

confidence: 99%

See 3 more Smart Citations

Transfer Learning in Magnetic Resonance Brain Imaging: A Systematic Review

et al. 2021

View full text Add to dashboard Cite

(1) Background: Transfer learning refers to machine learning techniques that focus on acquiring knowledge from related tasks to improve generalization in the tasks of interest. In magnetic resonance imaging (MRI), transfer learning is important for developing strategies that address the variation in MR images from different imaging protocols or scanners. Additionally, transfer learning is beneficial for reutilizing machine learning models that were trained to solve different (but related) tasks to the task of interest. The aim of this review is to identify research directions, gaps in knowledge, applications, and widely used strategies among the transfer learning approaches applied in MR brain imaging; (2) Methods: We performed a systematic literature search for articles that applied transfer learning to MR brain imaging tasks. We screened 433 studies for their relevance, and we categorized and extracted relevant information, including task type, application, availability of labels, and machine learning methods. Furthermore, we closely examined brain MRI-specific transfer learning approaches and other methods that tackled issues relevant to medical imaging, including privacy, unseen target domains, and unlabeled data; (3) Results: We found 129 articles that applied transfer learning to MR brain imaging tasks. The most frequent applications were dementia-related classification tasks and brain tumor segmentation. The majority of articles utilized transfer learning techniques based on convolutional neural networks (CNNs). Only a few approaches utilized clearly brain MRI-specific methodology, and considered privacy issues, unseen target domains, or unlabeled data. We proposed a new categorization to group specific, widely-used approaches such as pretraining and fine-tuning CNNs; (4) Discussion: There is increasing interest in transfer learning for brain MRI. Well-known public datasets have clearly contributed to the popularity of Alzheimer’s diagnostics/prognostics and tumor segmentation as applications. Likewise, the availability of pretrained CNNs has promoted their utilization. Finally, the majority of the surveyed studies did not examine in detail the interpretation of their strategies after applying transfer learning, and did not compare their approach with other transfer learning approaches.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

See 2 more Smart Citations

Transfer Learning in Magnetic Resonance Brain Imaging: A Systematic Review

et al. 2021

View full text Add to dashboard Cite

show abstract

“…It is assumed, that low-level features should be shared across domains, while high-level ones are more prone to domain shift and should be therefore fine-tuned. However, in a number of Domain Adaptation papers the presence of low-level domain shift is demonstrated [3,12,21].…”

Section: Introductionmentioning

confidence: 99%

Anatomy of Domain Shift Impact on U-Net Layers in MRI Segmentation

Zakazov¹,

Shirokikh²,

Chernyavskiy³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Domain Adaptation (DA) methods are widely used in medical image segmentation tasks to tackle the problem of differently distributed train (source) and test (target) data. We consider the supervised DA task with a limited number of annotated samples from the target domain. It corresponds to one of the most relevant clinical setups: building a sufficiently accurate model on the minimum possible amount of annotated data. Existing methods mostly fine-tune specific layers of the pretrained Convolutional Neural Network (CNN). However, there is no consensus on which layers are better to fine-tune, e.g. the first layers for images with low-level domain shift or the deeper layers for images with high-level domain shift. To this end, we propose SpotTUnet -a CNN architecture that automatically chooses the layers which should be optimally fine-tuned. More specifically, on the target domain, our method additionally learns the policy that indicates whether a specific layer should be fine-tuned or reused from the pretrained network. We show that our method performs at the same level as the best of the nonflexible fine-tuning methods even under the extreme scarcity of annotated data. Secondly, we show that SpotTUnet policy provides a layer-wise visualization of the domain shift impact on the network, which could be further used to develop robust domain generalization methods. In order to extensively evaluate SpotTUnet performance, we use a publicly available dataset of brain MR images (CC359), characterized by explicit domain shift. We release a reproducible experimental pipeline 3 .

show abstract