An empirical comparison of different approaches for combining multimodal neuroimaging data with support vector machine

Abstract

In the pursuit of clinical utility, neuroimaging researchers of psychiatric and neurological illness are increasingly using analyses, such as support vector machine, that allow inference at the single-subject level. Recent studies employing single-modality data, however, suggest that classification accuracies must be improved for such utility to be realized. One possible solution is to integrate different data types to provide a single combined output classification; either by generating a single decision function based on an integrated kernel matrix, or, by creating an ensemble of multiple single modality classifiers and integrating their predictions. Here, we describe four integrative approaches: (1) an un-weighted sum of kernels, (2) multi-kernel learning, (3) prediction averaging, and (4) majority voting, and compare their ability to enhance classification accuracy relative to the best single-modality classification accuracy. We achieve this by integrating structural, functional, and diffusion tensor magnetic resonance imaging data, in order to compare ultra-high risk (n = 19), first episode psychosis (n = 19) and healthy control subjects (n = 23). Our results show that (i) whilst integration can enhance classification accuracy by up to 13%, the frequency of such instances may be limited, (ii) where classification can be enhanced, simple methods may yield greater increases relative to more computationally complex alternatives, and, (iii) the potential for classification enhancement is highly influenced by the specific diagnostic comparison under consideration. In conclusion, our findings suggest that for moderately sized clinical neuroimaging datasets, combining different imaging modalities in a data-driven manner is no “magic bullet” for increasing classification accuracy. However, it remains possible that this conclusion is dependent on the use of neuroimaging modalities that had little, or no, complementary information to offer one another, and that the integration of more diverse types of data would have produced greater classification enhancement. We suggest that future studies ideally examine a greater variety of data types (e.g., genetic, cognitive, and neuroimaging) in order to identify the data types and combinations optimally suited to the classification of early stage psychosis.

Multimodal Fusion of Brain Imaging Data: A Key to Finding the Missing Link(s) in Complex Mental Illness

“…By adopting Gaussian process classifiers to evaluate the prognostic value of neuroimaging data and clinical characteristics, Schmaal et al(99) discovered that prediction of the naturalistic course of depression over 2 years is improved by considering different task contrasts or data sources, especially those derived from neural responses to emotional facial expressions. Finally, Pettersson-Yeo et al(100) used a multimodal SVM approach to examine the ability of sMRI, fMRI, dMRI and cognitive data to differentiate between ultra-high-risk (UHR) and first-episode (FEP) psychosis at the single-subject level, supporting clinical development of SVM to help inform identification of FEP and UHR. These findings strongly suggest that multi-modal classification facilitated by advanced modeling techniques can provide more accurate and early detection of brain abnormalities beyond approaches that use only a single modality.…”

Using neuroimaging to help predict the onset of psychosis

“…A feature of machine learning methods that has yet to be fully exploited in the UHR field is that their use is not restricted to neuroimaging data: they can be applied to multiple data modalities 86 . The integration of data from different modalities may be particularly useful in predicting the onset of psychosis, as it is the result of interactions between a diversity of genetic, environmental and neurobiological factors 87 .…”

“…To date, most SVM studies in UHR subjects have involved data from a single neuroimaging modality (MRI), although the impact of introducing additional data modalities has recently been explored 86 . This partly reflects the logistical demands associated with acquiring multi-modal neuroimaging data in samples that are large enough to permit comparison of subgroups with different clinical outcomes 95 .…”

Structural and metabolic differentiation between bipolar disorder with psychosis and substance-induced psychosis: An integrated MRI/PET study

“…1 and 2; for PET results see and SIP patients showed a wider temporal alterations encompass-298 ing the superior, middle and inferior temporal gyri. In this regard, it 299 should be mentioned that the superior temporal gyrus is involved 300 in auditory and language processing [31] and theory of mind [32], 301 abilities that are notably abnormal in individuals with psychosis 302 [31,[33][34][35][36]. Therefore, it is plausible that this common brain 303 dysfunction in the superior temporal gyrus, transversally present 304 in all the three groups of patients with psychosis, could be 305 considered a biological marker of psychosis [37].…”