Robust Attribute and Structure Preserving Graph Embedding

Abstract

Graph embedding methods are useful for a wide range of graph analysis tasks including link prediction and node classification. Most graph embedding methods learn only the topological structure of graphs. Nevertheless, it has been shown that the incorporation of node attributes is beneficial in improving the expressive power of node embeddings. However, real-world graphs are often noisy in terms of structure and/or attributes (missing and/or erroneous edges/attributes). Most existing graph embedding methods are susceptible to this noise, as they do not consider uncertainty during the modelling process. In this paper, we introduce RASE, a Robust Attribute and Structure preserving graph Embedding model. RASE is a novel graph representation learning model which effectively preserves both graph structure and node attributes through a unified loss function. To be robust, RASE uses a denoising attribute auto-encoder to deal with node attribute noise, and models uncertainty in the embedding space as Gaussians to cope with graph structure noise. We evaluate the performance of RASE through an extensive experimental study on various real-world datasets. Results demonstrate that RASE outperforms state-of-the-art embedding methods on multiple graph analysis tasks and is robust to both structure and attribute noise.

Survey on graph embeddings and their applications to machine learning problems on graphs

“…Liu (2020) builds Anonymized GCN with adversarial training to be robust to the noise attacks. Hettige et al (2020) propose the RASE model, that applies Gaussian denoising attribute autoencoder for achieving robustness of received embedding, while Laakom et al (2020) Deep Learning models are now a study of vulnerability to adversarial attacks, in particular, it relates to structural data. The first approaches for detection of node/edge add/remove mechanisms were studied in Bojcheski & Günnemann (2018), Chen et al (2018c), while other researchers focused on methods for unsupervised (Sun et al, 2018b), semi-supervised (Chen et al, 2018e) and supervised (Zügner, Akbarnejad & Günnemann, 2018) scenarios of graph embedding construction, and application for ML problems.…”

“…Hettige et al (2020) propose the RASE model, that applies Gaussian denoising attribute autoencoder for achieving robustness of received embedding, whileLaakom et al (2020) catches the uncertainty by learning probability Gaussian distributions over embedding space. Weng, Zhang & Dou (2020) employs adversarial training for variational graph autoencoder.…”

$\mathtt{MedGraph:}$ Structural and Temporal Representation Learning of Electronic Medical Records

“…Embedding into a common L-dimensional embedding space enables similarity computation between two heterogeneous nodes. Since the embeddings are Gaussians, we measure the Wasserstein distance as in DVNE [37] and RASE [16], specifically 2-nd Wasserstein distance (W2) between the embeddings. By computing W2 distance, we can preserve transitivity property in the embedding space [37].…”