Fast Network Embedding Enhancement via High Order Proximity Approximation

Yang, Cheng; Sun, Maosong; Liu, Zhiyuan; Tu, Cunchao

doi:10.24963/ijcai.2017/544

Cited by 150 publications

(98 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The representative works are DeepWalk [23] and node2vec [8], which use first-order and second-order random walks to guide the network embedding respectively. Later a lot of analysis was conducted to build the relationship between the random walk based methods and the matrix factorization based methods, which shows the fact that DeepWalk uses the embedding vectors to learn from the truncated average commuting time between nodes [5,31]. This also implies that when performing random walk, to obtain a set of stable embedding vectors, one needs to run sufficient steps of random walk to estimate the stationary distributions of hitting time and commuting time.…”

Section: Related Workmentioning

confidence: 99%

“…For example, in DeepWalk [23], it first samples random walk paths from the network following the transition probabilities. Then it performs the Skip-gram algorithm [19] on the random walk paths to learn node embeddings to preserve the truncated average commuting time between nodes [5,31]. Node2vec [8] further generalizes this idea to introduce a second-order random walk to balance the breadth and depth of the search to explore neighborhoods of nodes.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Vertex-reinforced Random Walk for Network Embedding

Xiao

Zhao²,

Zheng³

et al. 2020

Proceedings of the 2020 SIAM International Conference on Data Mining

View full text Add to dashboard Cite

In this paper, we study the fundamental problem of random walk for network embedding. We propose to use non-Markovian random walk, variants of vertex-reinforced random walk (VRRW), to fully use the history of a random walk path. To solve the getting stuck problem of VRRW, we introduce an exploitation-exploration mechanism to help the random walk jump out of the stuck set. The new random walk algorithms share the same convergence property of VRRW and thus can be used to learn stable network embeddings. Experimental results on two link prediction benchmark datasets and three node classification benchmark datasets show that our proposed approach reinforce2vec can outperform state-of-the-art random walk based embedding methods by a large margin.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Vertex-reinforced Random Walk for Network Embedding

Xiao

Zhao²,

Zheng³

et al. 2020

Proceedings of the 2020 SIAM International Conference on Data Mining

View full text Add to dashboard Cite

show abstract

“…For example, LP [22] and ICA [15] estimate labels of unlabeled nodes using the local inference. Recently, graph embedding methods have been proposed to learn low-dimensional representations of nodes by leveraging many relational properties such as random walk (e.g., Node2Vec [8]), high-order paths (e.g., GraRep [2] and NEU [19]), structural similarities (e.g., Struc2Vec [13]).…”

Section: Related Workmentioning

confidence: 99%

Role Equivalence Attention for Label Propagation in Graph Neural Networks

Park

Neville

2020

Advances in Knowledge Discovery and Data Mining

View full text Add to dashboard Cite

Semi-supervised relational learning methods aim to classify nodes in a partially-labeled graph. While popular, existing methods using Graph Neural Networks (GNN) for semi-supervised relational learning have mainly focused on learning node representations by aggregating nearby attributes, and it is still challenging to leverage inferences about unlabeled nodes with few attributes-particularly when trying to exploit higher-order relationships in the network efficiently. To address this, we propose a Graph Neural Network architecture that incorporates patterns among the available class labels and uses (1) a Role Equivalence attention mechanism and (2) a mini-batch importance sampling method to improve efficiency when learning over high-order paths. In particular, our Role Equivalence attention mechanism is able to use nodes' roles to learn how to focus on relevant distant neighbors, in order to adaptively reduce the increased noise that occurs when higher-order structures are considered. In experiments on six different real-world datasets, we show that our model (REGNN) achieves significant performance gains compared to other recent state-of-the-art baselines, particularly when higher-order paths are considered in the models.

show abstract

“…Due to its ability in facilitating graph analysis, graph representation learning has drawn researchers' attentions from machine learning and data mining fields [5]- [7]. Most of these works are focusing on static networks, such as explicitly preserving local or high-order proximity [8]- [10], learning representations using truncated walks [11], [12]; using matrix factorization technique to obtain latent vectors [6], [13], [14], incorporating heterogeneous information [5], [7], etc. Most of these methods are lack of theoretical support to produce reliable representation results.…”

Section: Extraction Decouplingmentioning

confidence: 99%

FI-GRL: Fast Inductive Graph Representation Learning via Projection-Cost Preservation

Jiang

Zheng

et al. 2018

2018 IEEE International Conference on Data Mining (ICDM)

View full text Add to dashboard Cite

Graph representation learning aims at transforming graph data into meaningful low-dimensional vectors to facilitate the employment of machine learning and data mining algorithms designed for general data. Most current graph representation learning approaches are transductive, which means that they require all the nodes in the graph are known when learning graph representations and these approaches cannot naturally generalize to unseen nodes. In this paper, we present a Fast Inductive Graph Representation Learning framework (FI-GRL) to learn nodes' low-dimensional representations. Our approach can obtain accurate representations for seen nodes with provable theoretical guarantees and can easily generalize to unseen nodes. Specifically, in order to explicitly decouple nodes' relations expressed by the graph, we transform nodes into a randomized subspace spanned by a random projection matrix. This stage is guaranteed to preserve the projection-cost of the normalized random walk matrix which is highly related to the normalized cut of the graph. Then feature extraction is achieved by conducting singular value decomposition on the obtained matrix sketch. By leveraging the property of projection-cost preservation on the matrix sketch, the obtained representation result is nearly optimal. To deal with unseen nodes, we utilize folding-in technique to learn their meaningful representations. Empirically, when the amount of seen nodes are larger than that of unseen nodes, FI-GRL always achieves excellent results. Our algorithm is fast, simple to implement and theoretically guaranteed. Extensive experiments on real datasets demonstrate the superiority of our algorithm on both efficacy and efficiency over both macroscopic level (clustering) and microscopic level (structural hole detection) applications.

show abstract

Fast Network Embedding Enhancement via High Order Proximity Approximation

Cited by 150 publications

References 20 publications

Vertex-reinforced Random Walk for Network Embedding

Vertex-reinforced Random Walk for Network Embedding

Role Equivalence Attention for Label Propagation in Graph Neural Networks

FI-GRL: Fast Inductive Graph Representation Learning via Projection-Cost Preservation

Contact Info

Product

Resources

About