This paper introduces Graph Convolutional Recurrent Network (GCRN), a deep learning model able to predict structured sequences of data. Precisely, GCRN is a generalization of classical recurrent neural networks (RNN) to data structured by an arbitrary graph. Such structured sequences can represent series of frames in videos, spatio-temporal measurements on a network of sensors, or random walks on a vocabulary graph for natural language modeling. The proposed model combines convolutional neural networks (CNN) on graphs to identify spatial structures and RNN to find dynamic patterns. We study two possible architectures of GCRN, and apply the models to two practical problems: predicting moving MNIST data, and modeling natural language with the Penn Treebank dataset. Experiments show that exploiting simultaneously graph spatial and dynamic information about data can improve both precision and learning speed. arXiv:1612.07659v1 [stat.ML] 22 Dec 2016 Under review as a conference paper at ICLR 2017 Figure 1: Illustration of the proposed GCRN model for spatio-temporal prediction of graph-structured data. The technique combines at the same time CNN on graphs and RNN. RNN can be easily exchanged with LSTM or GRU networks.

Convolutional Neural Networks (CNNs) are a cornerstone of the Deep Learning toolbox and have led to many breakthroughs in Artificial Intelligence. So far, these neural networks (NNs) have mostly been developed for regular Euclidean domains such as those supporting images, audio, or video. Because of their success, CNN-based methods are becoming increasingly popular in Cosmology. Cosmological data often comes as spherical maps, which make the use of the traditional CNNs more complicated. The commonly used pixelization scheme for spherical maps is the Hierarchical Equal Area isoLatitude Pixelisation (HEALPix). We present a spherical CNN for analysis of full and partial HEALPix maps, which we call DeepSphere. The spherical CNN is constructed by representing the sphere as a graph. Graphs are versatile data structures that can represent pairwise relationships between objects or act as a discrete representation of a continuous manifold. Using the graph-based representation, we define many of the standard CNN operations, such as convolution and pooling. With filters restricted to being radial, our convolutions are equivariant to rotation on the sphere, and DeepSphere can be made invariant or equivariant to rotation. This way, DeepSphere is a special case of a graph CNN, tailored to the HEALPix sampling of the sphere. This approach is computationally more efficient than using spherical harmonics to perform convolutions. We demonstrate the method on a classification problem of weak lensing mass maps from two cosmological models and compare its performance with that of three baseline classifiers, two based on the power spectrum and pixel density histogram, and a classical 2D CNN. Our experimental results show that the performance of DeepSphere is always superior or equal to the baselines. For high noise levels and for data covering only a smaller fraction of the sphere, DeepSphere achieves typically 10% better classification accuracy than the baselines. Finally, we show how learned filters can be visualized to introspect the NN.Code and examples are available at https://github.com/SwissDataScienceCenter/DeepSphere.

No abstract

No abstract

We present an approach for tracking fast spatiotemporal cortical dynamics in which we combine white matter connectivity data with source-projected electroencephalographic (EEG) data. We employ the mathematical framework of graph signal processing in order to derive the Fourier modes of the brain structural connectivity graph, or "network harmonics" . These network harmonics are naturally ordered by smoothness. Smoothness in this context can be understood as the amount of variation along the cortex, leading to a multi-scale representation of brain connectivity. We demonstrate that network harmonics provide a sparse representation of the EEG signal, where, at certain times, the smoothest 15 network harmonics capture 90% of the signal power. This suggests that network harmonics are functionally meaningful, which we demonstrate by using them as a basis for the functional EEG data recorded from a face detection task. There, only 13 network harmonics are sufficient to track the large-scale cortical activity during the processing of the stimuli with a 50 ms resolution, reproducing well-known activity in the fusiform face area as well as revealing co-activation patterns in somatosensory/motor and frontal cortices that an unconstrained ROI-by-ROI analysis fails to capture. The proposed approach is simple and fast, provides a means of integration of multimodal datasets, and is tied to a theoretical framework in mathematics and physics. Thus, network harmonics point towards promising research directions both theoretically -for example in exploring the relationship between structure and function in the brain -and practically -for example for network tracking in different tasks and groups of individuals, such as patients.