Factorised Spatial Representation Learning: Application in Semi-supervised Myocardial Segmentation

Chartsias, Agisilaos; Joyce, T. A.; Papanastasiou, Giorgos; Semple, Scott; Williams, Michelle C.; Newby, David E.; Dharmakumar, Rohan; Tsaftaris, Sotirios A.

doi:10.1007/978-3-030-00934-2_55

Cited by 60 publications

(49 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The decomposition process yields representations for the anatomy and the modality characteristics of medical images and is achieved by two dedicated neural networks. Whilst a decomposition could also be performed with a single neural network with two separate outputs and shared layer components, as done in our previous work [Chartsias et al, 2018], we found that by using two separate networks, as also done in and in Lee et al [2018], we can more easily control the information captured by each factor, and we can stabilise the behaviour of each encoder during training.…”

Section: Input Decompositionmentioning

confidence: 96%

See 1 more Smart Citation

Disentangled representation learning in cardiac image analysis

Chartsias

Joyce

Papanastasiou³

et al. 2019

Medical Image Analysis

Self Cite

162

143

View full text Add to dashboard Cite

Typically, a medical image offers spatial information on the anatomy (and pathology) modulated by imaging specific characteristics. Many imaging modalities including Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) can be interpreted in this way. We can venture further and consider that a medical image naturally factors into some spatial factors depicting anatomy and factors that denote the imaging characteristics. Here, we explicitly learn this decomposed (disentangled) representation of imaging data, focusing in particular on cardiac images. We propose Spatial Decomposition Network (SDNet), which factorises 2D medical images into spatial anatomical factors and non-spatial modality factors. We demonstrate that this high-level representation is ideally suited for several medical image analysis tasks, such as semi-supervised segmentation, multi-task segmentation and regression, and image-to-image synthesis. Specifically, we show that our model can match the performance of fully supervised segmentation models, using only a fraction of the labelled images. Critically, we show that our factorised representation also benefits from supervision obtained either when we use auxiliary tasks to train the model in a multi-task setting (e.g. regressing to known cardiac indices), or when aggregating multimodal data from different sources (e.g. pooling together MRI and CT data). To explore the properties of the learned factorisation, we perform latent-space arithmetic and show that we can synthesise CT from MR and vice versa, by swapping the modality factors. We also demonstrate that the factor holding image specific information can be used to predict the input modality with high accuracy. Code will be made available at https://github. com/agis85/anatomy_modality_decomposition.

show abstract

Section: Input Decompositionmentioning

confidence: 96%

“…It is interesting to compare the performance of SDNet with our previous work [Chartsias et al, 2018]. We therefore modify our previous model for multi-class segmentation and repeat the experiment for the ACDC dataset.…”

Section: Semi-supervised Segmentationmentioning

confidence: 99%

Disentangled representation learning in cardiac image analysis

Chartsias

Joyce

Papanastasiou³

et al. 2019

Medical Image Analysis

Self Cite

162

143

View full text Add to dashboard Cite

show abstract

“…Unsupervised task Bai et al (2017) Embedding consistency Zhang et al (2017b) Image classification Sedai et al (2017) Image reconstruction Baur et al (2017) Manifold learning Chartsias et al (2018) Image reconstruction Huo et al (2018a) Image synthesis Zhao et al (2019) Image registration Li et al (2019) Transformation consistency the same-class pixels as close as possible while pushing apart the feature embedding of the pixels from different classes. To identify same-class pixels between labeled and unlabeled images, the authors assume the availability of a noisy label prior for unlabeled images.…”

Section: Publicationmentioning

confidence: 99%

“…For the task of gland segmentation in histopathology images, the authors have demonstrated one point increase in Dice over fully-supervised models trained with labeled data. Chartsias et al (2018) propose a solution to the problem of domain shift based on a disentangled image representation where the idea is to separate information related to segmenting the structure of interest from the other image features that readily change from one domain to another. By doing so, the segmentation network focuses on the intrinsic features of the target structure rather than variations related to imaging scanners or artifacts.…”

Section: Publicationmentioning

confidence: 99%

Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation

Tajbakhsh¹,

Jeyaseelan²,

Li³

et al. 2020

Medical Image Analysis

724

358

View full text Add to dashboard Cite

The medical imaging literature has witnessed remarkable progress in high-performing segmentation models based on convolutional neural networks. Despite the new performance highs, the recent advanced segmentation models still require large, representative, and high quality annotated datasets. However, rarely do we have a perfect training dataset, particularly in the field of medical imaging, where data and annotations are both expensive to acquire. Recently, a large body of research has studied the problem of medical image segmentation with imperfect datasets, tackling two major dataset limitations: scarce annotations where only limited annotated data is available for training, and weak annotations where the training data has only sparse annotations, noisy annotations, or image-level annotations. In this article, we provide a detailed review of the solutions above, summarizing both the technical novelties and empirical results. We further compare the benefits and requirements of the surveyed methodologies and provide our recommended solutions to the problems of scarce and weak annotations. We hope this review increases the community awareness of the techniques to handle imperfect datasets.

show abstract

“…An increased interest in SSL has also been seen in medical image anlaysis. The use of an unsupervised representation learning for better generalization has been investigated for the task of myocardial segmentation [2]. In [10], SSL was used in a similar X-ray data set, although the scope was limited to binary classifications between normal and abnormal categories.…”

Section: Related Workmentioning

confidence: 99%

Semi-supervised Learning by Disentangling and Self-ensembling over Stochastic Latent Space

Gyawali

Ghimire

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

The success of deep learning in medical imaging is mostly achieved at the cost of a large labeled data set. Semi-supervised learning (SSL) provides a promising solution by leveraging the structure of unlabeled data to improve learning from a small set of labeled data. Self-ensembling is a simple approach used in SSL to encourage consensus among ensemble predictions of unknown labels, improving generalization of the model by making it more insensitive to the latent space. Currently, such ensemble is obtained by randomization such as dropout regularization and random data augmentation. In this work, we hypothesize -from the generalization perspective -that self-ensembling can be improved by exploiting the stochasticity of a disentangled latent space. To this end, we present a stacked SSL model that utilizes unsupervised disentangled representation learning as the stochastic embedding for self-ensembling. We evaluate the presented model for multi-label classification using chest X-ray images, demonstrating its improved performance over related SSL models as well as the interpretability of its disentangled representations.

show abstract

Factorised Spatial Representation Learning: Application in Semi-supervised Myocardial Segmentation

Cited by 60 publications

References 17 publications

Disentangled representation learning in cardiac image analysis

Disentangled representation learning in cardiac image analysis

Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation

Semi-supervised Learning by Disentangling and Self-ensembling over Stochastic Latent Space

Contact Info

Product

Resources

About