With the advancement of brain imaging techniques and a variety of machine learning methods, significant progress has been made in brain disorder diagnosis, in particular Autism Spectrum Disorder. The development of machine learning models that can differentiate between healthy subjects and patients is of great importance. Recently, graph neural networks have found increasing application in domains where the population’s structure is modeled as a graph. The application of graphs for analyzing brain imaging datasets helps to discover clusters of individuals with a specific diagnosis. However, the choice of the appropriate population graph becomes a challenge in practice, as no systematic way exists for defining it. To solve this problem, we propose a population graph-based multi-model ensemble, which improves the prediction, regardless of the choice of the underlying graph. First, we construct a set of population graphs using different combinations of imaging and phenotypic features and evaluate them using Graph Signal Processing tools. Subsequently, we utilize a neural network architecture to combine multiple graph-based models. The results demonstrate that the proposed model outperforms the state-of-the-art methods on Autism Brain Imaging Data Exchange (ABIDE) dataset.

No abstract

With the advent of brain imaging techniques and machine learning tools, much effort has been devoted to building computational models to capture the encoding of visual information in the human brain. One of the most challenging brain decoding tasks is the accurate reconstruction of the perceived natural images from brain activities measured by functional magnetic resonance imaging (fMRI). In this work, we survey the most recent deep learning methods for natural image reconstruction from fMRI. We examine these methods in terms of architectural design, benchmark datasets, and evaluation metrics and present a fair performance evaluation across standardized evaluation metrics. Finally, we discuss the strengths and limitations of existing studies and present potential future directions.

No abstract

The vulnerability of computational models to adversarial examples highlights the differences in the ways humans and machines process visual information. Motivated by human perception invariance in object recognition, we aim to incorporate human brain representations for training a neural network. We propose a multi-modal approach that integrates visual input and the corresponding encoded brain signals to improve the adversarial robustness of the model. We investigate the effects of visual attacks of various strengths on an image classification task. Our experiments show that the proposed multi-modal framework achieves more robust performance against the increasing amount of adversarial perturbation than the baseline methods. Remarkably, in a black-box setting, our framework achieves a performance improvement of at least 7.54% and 5.97% on the MNIST and CIFAR-10 datasets, respectively. Finally, we conduct an ablation study to justify the necessity and significance of incorporating visual brain representations.cortex. The image-fMRI dataset comprises N paired samplesThe goal of a multi-modal 313 classifier is to learn a mapping function f : X , V → Y . 314 In terms of architecture, image classifier is based on 315 ResNet18 architecture [64]. fMRI classifier adopts a 2-layer 316 deep neural network (DNN) with two fully connected lay-317 ers separated by a non-linear activation function. While the 318 convolutional network networks are considered effective for 319 extracting information from images, we find simple feed-320 forward DNNs sufficient for performing fMRI classification. 321 The visual information from the image and brain codes 322 from the fMRI is integrated by extracting the visual feature 323 vector f x from the image and fMRI feature vector f v . Specif-324 ically, these extracted features f x and f v correspond to penul-325 timate or pre-classification layer activations for image and 326 fMRI classifiers. Then, the multi-modal method integrates 327 the visual and fMRI feature vectors into a single vector via 328 concatenation f xv = f x f v . Finally, using the cross-entropy 329 loss function, a fully-connected layer with a softmax predicts 330 the output class probability from f xv . 331 D. IMPLEMENTATION DETAILS 332 We train our networks using NVIDIA TITAN RTX GPUs. 333 The encoder and decoder are trained with the Adam opti-334 mization for 100 epochs with the learning rate of 1e-4 and 335 1e-3, and batch size 32 and 50, respectively. The main multi-336 modal framework was trained for 50 epochs using learning 337 rate 1e-2 and batch size 64. We ran the PGD and MIM attacks 338 for 10 steps.