Investigation of Deep-Learning-Driven Identification of Multiple Sclerosis Patients Based on Susceptibility-Weighted Images Using Relevance Analysis

Abstract

The diagnosis of multiple sclerosis (MS) is usually based on clinical symptoms and signs of damage to the central nervous system, which is assessed using magnetic resonance imaging. The correct interpretation of these data requires excellent clinical expertise and experience. Deep neural networks aim to assist clinicians in identifying MS using imaging data. However, before such networks can be integrated into clinical workflow, it is crucial to understand their classification strategy. In this study, we propose to use a convolutional neural network to identify MS patients in combination with attribution algorithms to investigate the classification decisions. The network was trained using images acquired with susceptibility-weighted imaging (SWI), which is known to be sensitive to the presence of paramagnetic iron components and is routinely applied in imaging protocols for MS patients. Different attribution algorithms were used to the trained network resulting in heatmaps visualizing the contribution of each input voxel to the classification decision. Based on the quantitative image perturbation method, we selected DeepLIFT heatmaps for further investigation. Single-subject analysis revealed veins and adjacent voxels as signs for MS, while the population-based study revealed relevant brain areas common to most subjects in a class. This pattern was found to be stable across different echo times and also for a multi-echo trained network. Intensity analysis of the relevant voxels revealed a group difference, which was found to be primarily based on the T1w magnitude images, which are part of the SWI calculation. This difference was not observed in the phase mask data.

Multimodal magnetic resonance imaging analysis in the characteristics of Wilson’s disease: A case report and literature review

“…SWI takes advantage of differences in magnetic susceptibility between tissues to show deposition of paramagnetic material and is particularly sensitive to iron [ 17 ]. An increasing number of studies have pointed out a complex relationship between copper and iron metabolisms in WD [ 18 , 19 ].…”

Role of artificial intelligence in MS clinical practice

“…AI techniques have been applied to obtain an even earlier and more specific diagnosis of MS. The application of CNNs on T2-weighted ( Wang et al, 2018 , Zhang et al, 2018 ), fluid attenuated inversion recovery (FLAIR) ( Eitel et al, 2019 ), and susceptibility-weighted ( Lopatina et al, 2020 ) images was able to separate MS patients from HC with high accuracy (70–99%). Heatmaps were used to try to “back-engineer” these algorithms (i.e., decode how they work) and highlight features contributing to algorithm classification.…”

“…Heatmaps were used to try to “back-engineer” these algorithms (i.e., decode how they work) and highlight features contributing to algorithm classification. This strategy showed that DL extracts relevant information from the presence of lesions on T2 and FLAIR sequences (especially around the posterior ventricular horns and in the corpus callosum) ( Wang et al, 2018 , Zhang et al, 2018 ) , from brain tissue abnormalities beyond the presence of lesions on lesion-refilled FLAIR sequences (corpus callosum, fornix, and thalamus) ( Eitel et al, 2019 ) and from voxels around the anterior ventricular horns, the occipital cortex and the veins in susceptibility-weighted scans ( Lopatina et al, 2020 ). The recognition of some abnormalities, present in MRI and not included in the diagnostic criteria for MS so far, might guide future revisions of diagnostic criteria.…”

Explainable Deep Learning Methods in Medical Imaging Diagnosis: A Survey

“…However, when a masking process is used to salient only the lung's area, the produced heatmaps highlight only the relevant regions since the CNN attention is limited to the critical area for detecting pulmonary diseases (lung's area). Following the same procedures, Lopatina et al [73] used DeepLIFT attribution algorithm to investigate the decisions of a multiple sclerosis classification model, and Sayres et al [109] used Integrated Gradients to provide explanations for the task of predicting diabetic retinopathy from retinal fundus images.…”