Neural Sheaf Diffusion: A Topological Perspective on Heterophily and Oversmoothing in GNNs

Abstract

Cellular sheaves equip graphs with "geometrical" structure by assigning vector spaces and linear maps to nodes and edges. Graph Neural Networks (GNNs) implicitly assume a graph with a trivial underlying sheaf. This choice is reflected in the structure of the graph Laplacian operator, the properties of the associated diffusion equation, and the characteristics of the convolutional models that discretise this equation. In this paper, we use cellular sheaf theory to show that the underlying geometry of the graph is deeply linked with the performance of GNNs in heterophilic settings and their oversmoothing behaviour. By considering a hierarchy of increasingly general sheaves, we study how the ability of the sheaf diffusion process to achieve linear separation of the classes in the infinite time limit expands. At the same time, we prove that when the sheaf is non-trivial, discretised parametric diffusion processes have greater control than GNNs over their asymptotic behaviour. On the practical side, we study how sheaves can be learned from data. The resulting sheaf diffusion models have many desirable properties that address the limitations of classical graph diffusion equations (and corresponding GNN models) and obtain state-of-the-art results in heterophilic settings. Overall, our work provides new connections between GNNs and algebraic topology and would be of interest to both fields.

Multi-Duplicated Characterization of Graph Structures Using Information Gain Ratio for Graph Neural Networks

“…The maximum accuracies demonstrated by our own implementation and those reported in other papers are shown in bold. The rows marked with "*" indicate the accuracies from papers [16], [37], [40], [41]. Figure 5 shows the optimal parameter settings used in MSI-H2GCN-2, which exhibited the highest average accuracy, as shown in Table 2.…”

Zoo guide to network embedding

“…The magnetic Laplacian can be considered a special case of the connection Laplacian, which can be used for vector diffusion maps [196]. Moreover, the connection Laplacian is used for formulating Sheaf neural networks [197,198], which are a new generation of neural networks obtaining excellent performance on several machine learning tasks. Finally, the non-backtracking matrix that identifies non-backtracking cycles and efficiently detects the network communities [199], has been recently proposed for embedding oriented edges (non-backtracking embedding dimensions) [200].…”

Boundary informed inverse PDE problems on discrete Riemann surfaces

“…Among the more recent developments, we also found the constructions of Hansen and Gebhart [2020] and Bodnar et al [2022] remarkable in that they imported the concept of sheaves into the setting of graph neural networks. This seems to be a natural approach that has the potential to produce more flexible models.…”