Learn the New, Keep the Old: Extending Pretrained Models with New Anatomy and Images

Özdemir, Firat; Fuernstahl, Philipp; Göksel, Orçun

doi:10.1007/978-3-030-00937-3_42

Cited by 47 publications

(44 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…consecutive slices from a 3D volume. In order to account for both of these conditions (i) and (ii), we propose to first pick a large batch of samples for which the trained model is "confident" [9], and then to prune them with the aim of maximizing how well they represent the entire dataset [4].…”

Section: Keeping Informative Samplesmentioning

confidence: 99%

“…Therefore, for all incremental settings for a given hold-out experiment, the validation and test sets are exactly the same, allowing us to make direct comparison between IRs of that set. 13,32,56,89,27,43,70,16,41,97,10,73,12,48,86,29,94,6,67,66,36,17,50,35,8,96,28,20,82,26,63,14,25,4,18,39,9,79,7,65,37,90,57,100,55,44,51,68,47,69,62,98,80,42,59,49,99,58,76,33,95,…”

Section: E Dataset Evaluationmentioning

confidence: 99%

“…This, however, has been shown to cause catastrophic forgetting [6] on the previous dataset/classes. Class-incremental learning has the potential to address this issue, since the goal is to achieve high performance on both new and old class categories, while having a bound on or a relatively slow increase of computational cost and memory footprint [7]- [9]. The state-of-the-art approach for class-incremental learning for classification task, iCaRL [7], exploits the concept of knowledge distillation [10] to retain old-class classification performance.…”

Section: Introductionmentioning

confidence: 99%

“…In addition, we also demonstrate the benefit of keeping informative samples throughout a lifelong learning scenario. We have presented the preliminary results of this work in [9]. Our specific contributions herein are: 1) Analysis on the larger and publicly available SKI10 dataset, targeting the knee anatomy and related surgical interventions.…”

Section: Introductionmentioning

confidence: 99%

“…Holdout set 1:47, 100, 54,13, 45, 20,1, 68, 71, 55, 33, 38, 58, 53, 85, 60, 97, 84, 63, 40, 28, 34, 69, 90, 57, 42, 46, 62, 75, 77,7,10, 78,3, 24,5, 74,4, 26,15, 31, 29, 87, 92, 44, 16, 73, 88, 21, 56, 94, 89, 41, 51, 93,9, 17, 99, 18, 61, 19, 91, 43, 27, 11, 67, 37, 96, 83, 80, 98, 32, 22, 76, 82, 36, 30, 50, 59, 49, 48, 6, 95, 81, 2, 86, 25, 66, 39, 23, 35, 12, 64, 70, 72, 8, 14, 52, 79, 65 Holdout set 4: 12, 82, 42, 95, 54, 5, 3, 60, 4, 33, 89, 24, 21, 65, 35, 43, 17, 91, 50, 61, 2, 59, 81, 7, 34, 53, 87, 80, 62, 38, 32, 27, 77, 67, 79, 52, 100, 49, 98, 20, 22, 84, 48, 94, 85, 96, 9, 41, 29, 44, 39, 92, 31, 75, 66, 69, 45, 70, 99, 18, 46, 6, 97, 26, 83, 37, 11, 16, 88, 90, 68, 36, 57, 19, 8, 74, 93, 14, 64, 56, 55, 78, 23, 10, 63, 13, 47, 1, 15, 86, 40, 72, 30, 28, 76, 25, 51, 58, 71, 73 Holdout set 5: 84, 54, 71, 46, 45, 40, 23, 81, 11, 1, 19, 31, 74, 34, 91, 5, 77, 78,…”

mentioning

confidence: 99%

See 4 more Smart Citations

Extending pretrained segmentation networks with additional anatomical structures

Özdemir

Göksel

2019

Int J CARS

Self Cite

View full text Add to dashboard Cite

For comprehensive surgical planning with sophisticated patient-specific models, all relevant anatomical structures need to be segmented. This could be achieved using deep neural networks given sufficiently many annotated samples, however datasets of multiple annotated structures are often unavailable in practice and costly to procure. Therefore, being able to build segmentation models with datasets from different studies and centers in an incremental fashion is highly desirable. We propose a class-incremental framework for extending a deep segmentation network to new anatomical structures using a minimal incremental annotation set. Through distilling knowledge from the current network state, we overcome the need for a full retraining. We evaluate our methods on 100 MR volumes from SKI10 challenge with varying incremental annotation ratios. For 50% incremental annotations, our proposed method suffers less than 1% Dice score loss in retaining old-class performance, as opposed to 25% loss of conventional finetuning. Our framework inherently exploits transferable knowledge from previously trained structures to incremental tasks, demonstrated by results superior even to non-incremental training: In a single volume one-shot incremental learning setting, our method outperforms vanilla network performance by >11% in Dice. With the presented method, new anatomical structures can be learned while retaining performance for older structures, without a major increase in complexity and memory footprint; hence suitable for lifelong class-incremental learning. By leveraging information from older examples, a fraction of annotations can be sufficient for incrementally building comprehensive segmentation models. With our meta-method, a deep segmentation network is extended with only a minor addition per structure, thus can be applicable also for future network architectures.

show abstract

Section: Keeping Informative Samplesmentioning

confidence: 99%

Section: E Dataset Evaluationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Extending pretrained segmentation networks with additional anatomical structures

Özdemir

Göksel

2019

Int J CARS

Self Cite

View full text Add to dashboard Cite

show abstract

Active Learning for Segmentation by Optimizing Content Information for Maximal Entropy

Özdemir

Peng

Tanner

et al. 2018

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Segmentation is essential for medical image analysis tasks such as intervention planning, therapy guidance, diagnosis, treatment decisions. Deep learning is becoming increasingly prominent for segmentation, where the lack of annotations, however, often becomes the main limitation. Due to privacy concerns and ethical considerations, most medical datasets are created, curated, and allow access only locally. Furthermore, current deep learning methods are often suboptimal in translating anatomical knowledge between different medical imaging modalities. Active learning can be used to select an informed set of image samples to request for manual annotation, in order to best utilize the limited annotation time of clinical experts for optimal outcomes, which we focus on in this work. Our contributions herein are two fold: (1) we enforce domain-representativeness of selected samples using a proposed penalization scheme to maximize information at the network abstraction layer, and (2) we propose a Borda-count based sample querying scheme for selecting samples for segmentation. Comparative experiments with baseline approaches show that the samples queried with our proposed method, where both above contributions are combined, result in significantly improved segmentation performance for this active learning task.

show abstract

Incremental Learning of Fetal Heart Anatomies Using Interpretable Saliency Maps

Patra

Noble

2020

Communications in Computer and Information Science

View full text Add to dashboard Cite

While medical image analysis has seen extensive use of deep neural networks, learning over multiple tasks is a challenge for connectionist networks due to tendencies of degradation in performance over old tasks while adapting to novel tasks. It is pertinent that adaptations to new data distributions over time are tractable with automated analysis methods as medical imaging data acquisition is typically not a static problem. So, one needs to ensure that a continual learning paradigm be ensured in machine learning methods deployed for medical imaging. To explore interpretable lifelong learning for deep neural networks in medical imaging, we introduce a perspective of understanding forgetting in neural networks used in ultrasound image analysis through the notions of attention and saliency. Concretely, we propose quantification of forgetting as a decline in the quality of class specific saliency maps after each subsequent task schedule. We also introduce a knowledge transfer from past tasks to present by a saliency guided retention of past exemplars which improve the ability to retain past knowledge while optimizing parameters for current tasks. Experiments on a clinical fetal echocardiography dataset demonstrate a state-of-the-art performance for our protocols.

show abstract

Learn the New, Keep the Old: Extending Pretrained Models with New Anatomy and Images

Cited by 47 publications

References 10 publications

Extending pretrained segmentation networks with additional anatomical structures

Extending pretrained segmentation networks with additional anatomical structures

Active Learning for Segmentation by Optimizing Content Information for Maximal Entropy

Incremental Learning of Fetal Heart Anatomies Using Interpretable Saliency Maps

Contact Info

Product

Resources

About