Locally Private Graph Neural Networks

Abstract

Graph Neural Networks (GNNs) have become a popular tool for learning on graphs, but their widespread use raises privacy concerns as graph data can contain personal or sensitive information. Differentially private GNN models have been recently proposed to preserve privacy while still allowing for effective learning over graph-structured datasets. However, achieving an ideal balance between accuracy and privacy in GNNs remains challenging due to the intrinsic structural connectivity of graphs. In this paper, we propose a new differentially private GNN called ProGAP that uses a progressive training scheme to improve such accuracy-privacy trade-offs. Combined with the aggregation perturbation technique to ensure differential privacy, ProGAP splits a GNN into a sequence of overlapping submodels that are trained progressively, expanding from the first submodel to the complete model. Specifically, each submodel is trained over the privately aggregated node embeddings learned and cached by the previous submodels, leading to an increased expressive power compared to previous approaches while limiting the incurred privacy costs. We formally prove that ProGAP ensures edge-level and node-level privacy guarantees for both training and inference stages, and evaluate its performance on benchmark graph datasets. Experimental results demonstrate that ProGAP can achieve up to 5-10% higher accuracy than existing state-of-the-art differentially private GNNs.

Degree-Preserving Randomized Response for Graph Neural Networks under Local Differential Privacy

“…RELATED WORK DP on GNNs. In the past year or two, differentially private GNNs have become a very active research topic [23]- [30] (most of them are preprints). Most of them assume a centralized model [23]- [27] where the server has the entire graph (or the exact number of edges in the entire graph [25]).…”

“…Sajadmanesh and Gatica-Perez [30] apply LDP to each user's feature vectors in an attributed graph. However, they do not hide edges, which are sensitive in a social graph.…”

“…DP has been recently applied to GNNs [23]- [30], and most of them adopt centralized DP. However, they suffer from the data breach issue explained above.…”

“…LDP can be applied to these decentralized SNSs and does not suffer from data leakage. Sajadmanesh and Gatica-Perez [30] apply LDP to each user's feature vector (attribute values such as age, gender, city, and personal website) in GNNs. They focus on node classification and show that their algorithm provides high classification accuracy with a small privacy budget ε in LDP, e.g., ε = 1.…”

“…In this case, feature vectors in GNNs are generated from the graph structure [4]. Unfortunately, the algorithm in [30] cannot be used to protect user privacy for such applications.…”