Input Dropout for Spatially Aligned Modalities

Blois, Sébastien de; Garon, Mathieu; Gagné, Christian; Lalonde, Jean‐François

doi:10.1109/icip40778.2020.9190858

Cited by 7 publications

(1 citation statement)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A straightforward way to tackle this problem is to train independent models for each missing-modality condition but this requires a long training time. To overcome this issue, a training strategy termed Modality Dropout (ModDrop) [19] has been developed and widely used in various fields such as computer vision [4], dialogue systems [20] and medical imaging [17,21]. Particularly, for MS lesion segmentation, Feng et al [7] adopted ModDrop and achieved the state-of-theart performance to handle MRIs with missing sequences.…”

Section: Introductionmentioning

confidence: 99%

ModDrop++: A Dynamic Filter Network with Intra-subject Co-training for Multiple Sclerosis Lesion Segmentation with Missing Modalities

Liu¹,

Fan²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

Multiple Sclerosis (MS) is a chronic neuroinflammatory disease and multi-modality MRIs are routinely used to monitor MS lesions. Many automatic MS lesion segmentation models have been developed and have reached human-level performance. However, most established methods assume the MRI modalities used during training are also available during testing, which is not guaranteed in clinical practice. A training strategy termed Modality Dropout (ModDrop) has been applied to MS lesion segmentation to achieve the state-of-the-art performance for missing modality. We present a novel method dubbed ModDrop++ to train a unified network adaptive to an arbitrary number of input MRI sequences. Moreover, ModDrop++ can be easily applied to any existing model architectures. Specifically, ModDrop++ upgrades the main idea of ModDrop in two key ways. First, we devise a plug-and-play dynamic head and adopt a filter scaling strategy to improve the expressiveness of the network. Second, we design a co-training strategy to leverage the intrasubject relation between full modality and missing modality. In particular, the intra-subject co-training strategy aims to guide the dynamic head to generate similar feature representations between the full-and missingmodality data from the same subject. We use two public MS datasets to show the superiority of ModDrop++. Source code and trained models are available at https://github.com/han-liu/ModDropPlusPlus.

show abstract