Spatiotemporal Feature Selection Improves Prediction Accuracy of Multi-Voxel Pattern Classification

Abstract

Machine 33 34 Highlights: 35• Spatiotemporal feature selection effect on MVPC was assessed in slow event-related 36 fMRI 37• Spatiotemporal feature selection improved brain decoding accuracy 38• From ~2-11 seconds after stimuli onset were the most informative part of each trial 39• Random forest outperformed support vector machines 40• Random forest benefited more from temporal changes compared with support vector 41 machine 42 Abstract 43The importance of spatiotemporal feature selection in fMRI decoding studies has not 44 been studied exhaustively. Temporal embedding of features allows the incorporation of 45 brain activity dynamics into multivariate pattern classification, and may provide enriched 46 information about stimulus-specific response patterns and potentially improve prediction 47 accuracy. This study investigates the possibility of enhancing the classification 48 performance by exploring spatial and temporal (spatiotemporal) domain, to identify the 49 optimum combination of the spatiotemporal features based on the classification 50 performance. We investigated the importance of spatiotemporal feature selection using a 51 slow event-related design adapted from the classic Haxby et al. (2001) study. Data were 52 collected using a multiband fMRI sequence with temporal resolution of 0.568 seconds. A 53 wide range of spatiotemporal observations was created as various combinations of 54 spatiotemporal features. Using both random forest, and support vector machine, 55 classifiers, prediction accuracies for these combinations were then compared with the 56 single time-point spatial multivariate pattern approach that uses only a single temporal 57 observation. The results showed that on average spatiotemporal feature selection 58 improved prediction accuracy. Moreover, the random forest algorithm outperformed the 59 support vector machine and benefitted from temporal information to a greater extent. As 60 expected, the most influential temporal durations were found to be around the peak of the 61 hemodynamic response function, a few seconds after the stimuli onset until ~4 seconds 62 after the peak of the hemodynamic response function. The superiority of spatiotemporal 63 feature selection over single time-point spatial approaches invites future work to design 64 systematic and optimal approaches to the incorporation of spatiotemporal dependencies 65 into feature selection for decoding. 66 67 68shown to alter the results of MVPC in block design experiments (Choupan et al., 2014; 130 Mourao-Miranda, Friston, & Brammer, 2007; Sapountzis, Schluppeck, Bowtell, & 131 Peirce, 2010). 132Following the previous works, this study is based on the hypothesis that by embedding 133 the temporal dynamics provided by fMRI into the process of multivariate brain pattern 134 recognition, more information contained in the BOLD signal can be utilized compared 135 with single TR methods, leading to a potential improvement in the prediction 136 performance. In particular, we predicted that not all of the temporal dynamics...

Harnessing spatial homogeneity of neuroimaging data: patch individual filter layers for CNNs

“…Another method, aimed at improving learning in small sample sizes regimes, is to reduce the amount of features through selection (Choupan et al, 2019). A model parameter, such as a neural network filter, that is being trained on the entire input will be subject to a greater superimposition of different distributions (from signal and noise) than on a smaller selection of those features, i.e.…”