Locally Private Graph Neural Networks (2020-06-09T00:00:00.000000Z)

TL;DR

This paper proposes a privacy-preserving, architecture-agnostic GNN learning framework with formal privacy guarantees based on Local Differential Privacy (LDP), and develops a locally private mechanism to perturb and compress node features, which the server can efficiently collect to approximate the GNN's neighborhood aggregation step.

Abstract

Graph Neural Networks (GNNs) have demonstrated superior performance in learning node representations for various graph inference tasks. However, learning over graph data can raise privacy concerns when nodes represent people or human-related variables that involve sensitive or personal information. In this paper, we study the problem of node data privacy, where graph nodes (e.g., social network users) have potentially sensitive data that is kept private, but they could be beneficial for a central server for training a GNN over the graph. To address this problem, we propose a privacy-preserving, architecture-agnostic GNN learning framework with formal privacy guarantees based on Local Differential Privacy (LDP). Specifically, we develop a locally private mechanism to perturb and compress node features, which the server can efficiently collect to approximate the GNN's neighborhood aggregation step. Furthermore, to improve the accuracy of the estimation, we prepend to the GNN a denoising layer, called KProp, which is based on the multi-hop aggregation of node features. Finally, we propose a robust algorithm for learning with privatized noisy labels, where we again benefit from KProp's denoising capability to increase the accuracy of label inference for node classification. Extensive experiments conducted over real-world datasets demonstrate that our method can maintain a satisfying level of accuracy with low privacy loss.

Authors

D. Gática-Pérez

2 papers

Sina Sajadmanesh

1 papers

TL;DR

Abstract

Authors

References104 items

Unified Robust Training for Graph NeuralNetworks against Label Noise

When Machine Learning Meets Privacy

Model Extraction Attacks on Graph Neural Networks: Taxonomy and Realisation

The Surprising Power of Graph Neural Networks with Random Node Initialization

Quantifying Privacy Leakage in Graph Embedding

Graph Adversarial Networks: Protecting Information against Adversarial Attacks

Federated Dynamic GNN with Secure Aggregation

Adversarial Privacy-Preserving Graph Embedding Against Inference Attack

Local Differential Privacy and Its Applications: A Comprehensive Survey

More Than Privacy: Applying Differential Privacy in Key Areas of Artificial Intelligence

Learning From Noisy Labels With Deep Neural Networks: A Survey

Privacy-Preserving Graph Neural Network for Node Classification

Characteristic Functions on Graphs: Birds of a Feather, from Statistical Descriptors to Parametric Models

Stealing Links from Graph Neural Networks

Privacy in Deep Learning: A Survey

Unifying Graph Convolutional Neural Networks and Label Propagation

Advances and Open Problems in Federated Learning

SGNN: A Graph Neural Network Based Federated Learning Approach by Hiding Structure

Diffusion Improves Graph Learning

Multi-scale Attributed Node Embedding

Federated Learning: Challenges, Methods, and Future Directions

GraphSAINT: Graph Sampling Based Inductive Learning Method

Graph Embedding Matrix Sharing With Differential Privacy

MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing

Collecting and Analyzing Multidimensional Data with Local Differential Privacy

A Pragmatic Introduction to Secure Multi-Party Computation

Learning Graph Neural Networks with Noisy Labels

Probabilistic End-To-End Noise Correction for Learning With Noisy Labels

Fast Graph Representation Learning with PyTorch Geometric

Simplifying Graph Convolutional Networks

Graph Neural Networks for Social Recommendation

Privacy-Preserving Collaborative Deep Learning With Unreliable Participants

Graph Neural Networks: A Review of Methods and Applications

No Peek: A Survey of private distributed deep learning

A Review of Homomorphic Encryption Libraries for Secure Computation

Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks

How Powerful are Graph Neural Networks?

Predict then Propagate: Graph Neural Networks meet Personalized PageRank

Distributed learning of deep neural network over multiple agents

Representation Learning on Graphs with Jumping Knowledge Networks

DPNE: Differentially Private Network Embedding

Privacy-preserving Machine Learning as a Service

Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels

Locally Differentially Private Frequent Itemset Mining

Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoising

PrivTrie: Effective Frequent Term Discovery under Local Differential Privacy

Link Prediction Based on Graph Neural Networks

Hadamard Response: Estimating Distributions Privately, Efficiently, and with Little Communication

Communication Efficient, Sample Optimal, Linear Time Locally Private Discrete Distribution Estimation

Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning

Collecting Telemetry Data Privately

Hybrid Approach of Relation Network and Localized Graph Convolutional Filtering for Breast Cancer Subtype Classification

CryptoDL: Deep Neural Networks over Encrypted Data

Heavy Hitters and the Structure of Local Privacy

Marginal Release Under Local Differential Privacy

Graph Attention Networks

Practical Secure Aggregation for Privacy-Preserving Machine Learning

mixup: Beyond Empirical Risk Minimization

Representation Learning on Graphs: Methods and Applications

Privacy Loss in Apple's Implementation of Differential Privacy on MacOS 10.12

Local Private Hypothesis Testing: Chi-Square Tests

Locally Differentially Private Heavy Hitter Identification

Locally Differentially Private Protocols for Frequency Estimation

Clustering Algorithms for the Centralized and Local Models

Self-Normalizing Neural Networks

Inductive Representation Learning on Large Graphs

DeepSecure: Scalable Provably-Secure Deep Learning

Supervised Community Detection with Line Graph Neural Networks

Community Detection with Graph Neural Networks

Optimal Schemes for Discrete Distribution Estimation Under Locally Differential Privacy

Variational Graph Auto-Encoders

Heavy Hitter Estimation over Set-Valued Data with Local Differential Privacy

Federated Learning: Strategies for Improving Communication Efficiency

Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach

Semi-Supervised Classification with Graph Convolutional Networks

Mutual Information Optimally Local Private Discrete Distribution Estimation