PINE: Universal Deep Embedding for Graph Nodes via Partial Permutation Invariant Set Functions

Gui, Shupeng; Zhang, Xiangliang; Zhong, Pan; Qiu, Shuang; Wu, Mengyang; Ye, Jieping; Wang, Zhengdao; Liu, Ji

doi:10.1109/tpami.2021.3061162

Cited by 6 publications

(3 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The main idea of the paper [4] revolves around the problem of graph node embedding. The method proposed, coined PINE, introduces a novel notion of partial permutation invariant set function, to capture dependencies.…”

Section: Graph Neural Networkmentioning

confidence: 99%

Guest Editorial: Non-Euclidean Machine Learning

Zafeiriou

Bronstein

Cohen³

et al. 2022

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Section: Graph Neural Networkmentioning

confidence: 99%

Guest Editorial: Non-Euclidean Machine Learning

Zafeiriou

Bronstein

Cohen³

et al. 2022

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

“…PE transfers the Euclidean distance between two nodes in the embeding space to the probability of existence of an edge between them. PINE [77] can adaptively learn an arbitrary embedding function from node neighborhood via partial permutation invariant set function. It is applicable to learn node embeddings for both homogeneous and heterogeneous graphs.…”

Section: B Graph Embeddingsmentioning

confidence: 99%

Graph Kernels Based on Multi-scale Graph Embeddings

Ye¹,

Tian²,

Chen³

2022

Preprint

View full text Add to dashboard Cite

Graph kernels are conventional methods for computing graph similarities. However, most of the R-convolution graph kernels face two challenges: 1) They cannot compare graphs at multiple different scales, and 2) they do not consider the distributions of substructures when computing the kernel matrix. These two challenges limit their performances. To mitigate the two challenges, we propose a novel graph kernel called the Multiscale Path-pattern Graph kernel (MPG), at the heart of which is the multi-scale path-pattern node feature map. Each element of the path-pattern node feature map is the number of occurrences of a path-pattern around a node. A path-pattern is constructed by the concatenation of all the node labels in a path of a truncated BFS tree rooted at each node. Since the path-pattern node feature map can only compare graphs at local scales, we incorporate into it the multiple different scales of the graph structure, which are captured by the truncated BFS trees of different depth. We use the Wasserstein distance to compute the similarity between the multi-scale path-pattern node feature maps of two graphs, considering the distributions of substructures. We empirically validate MPG on various benchmark graph datasets and demonstrate that it achieves state-of-the-art performance.

show abstract

“…This impedes the designs like centrality, spatial, and edge encoding because each node in the brain network has the same degree and connects to every other node by a single hop. Second, in previous graph transformer models, eigenvalues and eigenvectors are commonly used as positional embeddings because they can provide identity and positional information for each node [15,26]. Nevertheless, in brain networks, the connection profile, which is defined as each node's corresponding row in the brain network adjacency matrix, is recognized as the most effective node feature [13].…”

Section: Introductionmentioning

confidence: 99%

Brain Network Transformer

Kan¹,

Dai²,

Cui³

et al. 2022

Preprint

View full text Add to dashboard Cite

Human brains are commonly modeled as networks of Regions of Interest (ROIs) and their connections for the understanding of brain functions and mental disorders. Recently, Transformer-based models have been studied over different types of data, including graphs, shown to bring performance gains widely. In this work, we study Transformer-based models for brain network analysis. Driven by the unique properties of data, we model brain networks as graphs with nodes of fixed size and order, which allows us to (1) use connection profiles as node features to provide natural and low-cost positional information and (2) learn pairwise connection strengths among ROIs with efficient attention weights across individuals that are predictive towards downstream analysis tasks. Moreover, we propose an ORTHONORMAL CLUSTERING READOUT operation based on selfsupervised soft clustering and orthonormal projection. This design accounts for the underlying functional modules that determine similar behaviors among groups of ROIs, leading to distinguishable cluster-aware node embeddings and informative graph embeddings. Finally, we re-standardize the evaluation pipeline on the only one publicly available large-scale brain network dataset of ABIDE, to enable meaningful comparison of different models. Experiment results show clear improvements of our proposed BRAIN NETWORK TRANSFORMER on both the public ABIDE and our restricted ABCD datasets. The implementation is available at https://github.com/Wayfear/BrainNetworkTransformer. Nowadays Transformer-based models have led a tremendous success in various downstream tasks across fields including natural language processing [56, 17] and computer vision [20, 10, 55]. Recent efforts have also emerged to apply Transformer-based designs to graph representation learning. GAT [57] firstly adapts the attention mechanism to graph neural networks (GNNs) but only considers the local structures of neighboring nodes. Graph Transformer [21] injects edge information into the attention mechanism and leverages the eigenvectors of each node as positional embeddings. SAN [40] further enhances the positional embeddings by considering both eigenvalues and eigenvectors and improves the attention mechanism by extending the attention from local to global structures.36th Conference on Neural Information Processing Systems (NeurIPS 2022).

show abstract

PINE: Universal Deep Embedding for Graph Nodes via Partial Permutation Invariant Set Functions

Cited by 6 publications

References 49 publications

Guest Editorial: Non-Euclidean Machine Learning

Guest Editorial: Non-Euclidean Machine Learning

Graph Kernels Based on Multi-scale Graph Embeddings

Brain Network Transformer

Contact Info

Product

Resources

About