Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methods

Abstract

Many widely used datasets for graph machine learning tasks have generally been homophilous, where nodes with similar labels connect to each other. Recently, new Graph Neural Networks (GNNs) have been developed that move beyond the homophily regime; however, their evaluation has often been conducted on small graphs with limited application domains. We collect and introduce diverse nonhomophilous datasets from a variety of application areas that have up to 384x more nodes and 1398x more edges than prior datasets. We further show that existing scalable graph learning and graph minibatching techniques lead to performance degradation on these non-homophilous datasets, thus highlighting the need for further work on scalable non-homophilous methods. To address these concerns, we introduce LINKX -a strong simple method that admits straightforward minibatch training and inference. Extensive experimental results with representative simple methods and GNNs across our proposed datasets show that LINKX achieves state-ofthe-art performance for learning on non-homophilous graphs. Our codes and data are available at https://github.com/CUAI/Non-Homophily-Large-Scale.

Analyzing GCN Aggregation on GPU

“…As described in the previous section, the performance of the GCN aggregation kernels can be influenced by the size of a feature vector per vertex and the fraction of non-zero elements in a feature vector (called feature density in this paper). Figure 3 exhibits the feature vector size (X-axis) and the corresponding feature density (Y-axis) of the 32 homogeneous graph datasets used for the prior GCN researches [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41]. As shown in the figure, the feature density of a graph dataset is relatively low as the feature vector size is large.…”

A Historical Survey of Advances in Transformer Architectures

“…In order to benchmark these works, a number of datasets have been utilized by the various works. A few of the commonly used datasets are BookCorpus [48], WMT 2014 [49], Wikipedia [50], C4 [22], ImageNet [51], and COCO [52].…”

“…An early work building upon the transformer model was that of Shaw et al [19], which simply involved extending the self-attention mechanism of transformers to efficiently consider representations of the relative positions or distances between sequence In order to benchmark these works, a number of datasets have been utilized by the various works. A few of the commonly used datasets are BookCorpus [48], WMT 2014 [49], Wikipedia [50], C4 [22], ImageNet [51], and COCO [52].…”