Multi-contrast Multi-scale vision Transformers (MMT) for MRI Contrast Synthesis

Abstract

Complementary information from multi-modal MRI is widely used in clinical practice for disease diagnosis. Due to scan time limitations, image corruptions, and different acquisition protocols, one or more contrasts may be missing or unusable. Recently developed CNN models for contrast synthesis are unable to capture the intricate dependencies between input contrasts and are not dynamic to the varying number of inputs. This work proposes a novel Multi-contrast and Multi-scale vision Transformer (MMT) that can take any number and combination of input sequences and synthesize the missing contrasts.