Releasing Graph Neural Networks with Differential Privacy Guarantees

Olatunji, Iyiola E.; Funke, Thorben; Khosla, Megha

doi:10.48550/arxiv.2109.08907

Cited by 4 publications

(10 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To protect the privacy of graph-structured data, many differential privacy preserving network embedding methods are investigated [136,152,208,228]. For instance, DPNE [208] applies perturbations on the objective function of learning network embeddings.…”

Section: Differentialmentioning

confidence: 99%

“…In [227], a perturbed gradient descent method that guarantees the privacy of graph embeddings learned by matrix factorization is proposed. More recently, several works [136,152] that focus on differentially private GNNs are explored. For example, the locally private GNN [152] adopts local differential privacy [92] to protect the privacy of node features by perturbing the user's features locally.…”

Section: Differentialmentioning

confidence: 99%

“…Furthermore, a robust graph convolution layer is investigated to reduce the negative effects of the injected noises. PrivGnn [136] extends the Private Aggregation of Teacher Ensembles (PATE) [141] for graph-structured data to release GNNs with differential privacy guarantees. In particularly, random subsampling on the training set of teacher and noisy labeling mechanism on the public data are used in PrivGNN [136] to achieve practical privacy guarantees.…”

Section: Differentialmentioning

confidence: 99%

“…PrivGnn [136] extends the Private Aggregation of Teacher Ensembles (PATE) [141] for graph-structured data to release GNNs with differential privacy guarantees. In particularly, random subsampling on the training set of teacher and noisy labeling mechanism on the public data are used in PrivGNN [136] to achieve practical privacy guarantees. As for [228], both the user feature perturbation at input stage and loss perturbation at optimization stage are investigated to achieve a privacy-preserving GNN for recommendation.…”

Section: Differentialmentioning

confidence: 99%

“…The privacy-preserving GNNs can be applied to address this concern. For instance, differential privacy-preserving GNNs [136,152,228] can be adopted in the pretraining phase to defend against the membership inference attack. Distributed Learning.…”

Section: Applications Of Privacy Preserving Gnnsmentioning

confidence: 99%

See 4 more Smart Citations

A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainability

Dai¹,

Zhao²,

Zhu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Graph Neural Networks (GNNs) have made rapid developments in the recent years. Due to their great ability in modeling graph-structured data, GNNs are vastly used in various applications, including high-stakes scenarios such as financial analysis, traffic predictions, and drug discovery. Despite their great potential in benefiting humans in the real world, recent study shows that GNNs can leak private information, are vulnerable to adversarial attacks, can inherit and magnify societal bias from training data and lack interpretability, which have risk of causing unintentional harm to the users and society. For example, existing works demonstrate that attackers can fool the GNNs to give the outcome they desire with unnoticeable perturbation on training graph. GNNs trained on social networks may embed the discrimination in their decision process, strengthening the undesirable societal bias. Consequently, trustworthy GNNs in various aspects are emerging to prevent the harm from GNN models and increase the users' trust in GNNs. In this paper, we give a comprehensive survey of GNNs in the computational aspects of privacy, robustness, fairness, and explainability. For each aspect, we give the taxonomy of the related methods and formulate the general frameworks for the multiple categories of trustworthy GNNs. We also discuss the future research directions of each aspect and connections between these aspects to help achieve trustworthiness.

show abstract

Section: Differentialmentioning

confidence: 99%

Section: Differentialmentioning

confidence: 99%

Section: Differentialmentioning

confidence: 99%

Section: Differentialmentioning

confidence: 99%

Section: Applications Of Privacy Preserving Gnnsmentioning

confidence: 99%

See 3 more Smart Citations

A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainability

Dai¹,

Zhao²,

Zhu³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

Locally Private Graph Neural Networks

Sajadmanesh

Gática-Pérez

2021

Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security

View full text Add to dashboard Cite

Graph Neural Networks (GNNs) have become a popular tool for learning on graphs, but their widespread use raises privacy concerns as graph data can contain personal or sensitive information. Differentially private GNN models have been recently proposed to preserve privacy while still allowing for effective learning over graph-structured datasets. However, achieving an ideal balance between accuracy and privacy in GNNs remains challenging due to the intrinsic structural connectivity of graphs. In this paper, we propose a new differentially private GNN called ProGAP that uses a progressive training scheme to improve such accuracy-privacy trade-offs. Combined with the aggregation perturbation technique to ensure differential privacy, ProGAP splits a GNN into a sequence of overlapping submodels that are trained progressively, expanding from the first submodel to the complete model. Specifically, each submodel is trained over the privately aggregated node embeddings learned and cached by the previous submodels, leading to an increased expressive power compared to previous approaches while limiting the incurred privacy costs. We formally prove that ProGAP ensures edge-level and node-level privacy guarantees for both training and inference stages, and evaluate its performance on benchmark graph datasets. Experimental results demonstrate that ProGAP can achieve up to 5-10% higher accuracy than existing state-of-the-art differentially private GNNs.

show abstract

Certified Graph Unlearning

Chien¹,

Pan²,

Milenković³

2022

Preprint

View full text Add to dashboard Cite

Graph-structured data is ubiquitous in practice and often processed using graph neural networks (GNNs). With the adoption of recent laws ensuring the "right to be forgotten", the problem of graph data removal has become of significant importance. To address the problem, we introduce the first known framework for certified graph unlearning of GNNs. In contrast to standard machine unlearning, new analytical and heuristic unlearning challenges arise when dealing with complex graph data. First, three different types of unlearning requests need to be considered, including node feature, edge and node unlearning. Second, to establish provable performance guarantees, one needs to address challenges associated with feature mixing during propagation. The underlying analysis is illustrated on the example of simple graph convolutions (SGC) and their generalized PageRank (GPR) extensions, thereby laying the theoretical foundation for certified unlearning of GNNs. Our empirical studies on six benchmark datasets demonstrate excellent performance-complexity trade-offs when compared to complete retraining methods and approaches that do not leverage graph information. For example, when unlearning 20% of the nodes on the Cora dataset, our approach suffers only a 0.1% loss in test accuracy while offering a 4-fold speed-up compared to complete retraining. Our scheme also outperforms unlearning methods that do not leverage graph information with a 12% increase in test accuracy for a comparable time complexity. Our code will be released upon acceptance. * Equal contribution Preprint. Under review.

show abstract

Releasing Graph Neural Networks with Differential Privacy Guarantees

Cited by 4 publications

References 19 publications

A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainability

A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainability

Locally Private Graph Neural Networks

Certified Graph Unlearning

Contact Info

Product

Resources

About