Advancements of federated learning towards privacy preservation: from federated learning to split learning (2020-11-25T00:00:00.000000Z)

TL;DR

This chapter is designed to provide extensive coverage in SL and its variants, including fundamentals, existing findings, integration with privacy measures such as differential privacy, open problems, and code implementation.

Abstract

In the distributed collaborative machine learning (DCML) paradigm, federated learning (FL) recently attracted much attention due to its applications in health, finance, and the latest innovations such as industry 4.0 and smart vehicles. FL provides privacy-by-design. It trains a machine learning model collaboratively over several distributed clients (ranging from two to millions) such as mobile phones, without sharing their raw data with any other participant. In practical scenarios, all clients do not have sufficient computing resources (e.g., Internet of Things), the machine learning model has millions of parameters, and its privacy between the server and the clients while training/testing is a prime concern (e.g., rival parties). In this regard, FL is not sufficient, so split learning (SL) is introduced. SL is reliable in these scenarios as it splits a model into multiple portions, distributes them among clients and server, and trains/tests their respective model portions to accomplish the full model training/testing. In SL, the participants do not share both data and their model portions to any other parties, and usually, a smaller network portion is assigned to the clients where data resides. Recently, a hybrid of FL and SL, called splitfed learning, is introduced to elevate the benefits of both FL (faster training/testing time) and SL (model split and training). Following the developments from FL to SL, and considering the importance of SL, this chapter is designed to provide extensive coverage in SL and its variants. The coverage includes fundamentals, existing findings, integration with privacy measures such as differential privacy, open problems, and code implementation.

Authors

Chandra Thapa

3 papers

Pathum Chamikara Mahawaga Arachchige

2 papers

S. Çamtepe

3 papers

TL;DR

Abstract

Authors

References58 items

Precision Health Data: Requirements, Challenges and Existing Techniques for Data Security and Privacy

Multiple Classification with Split Learning

NoPeek: Information leakage reduction to share activations in distributed deep learning

SplitNN-driven Vertical Partitioning

FedML: A Research Library and Benchmark for Federated Machine Learning

Federated Learning with Compression: Unified Analysis and Sharp Guarantees

SplitFed: When Federated Learning Meets Split Learning

Federated Learning with Only Positive Labels

End-to-End Evaluation of Federated Learning and Split Learning for Internet of Things

Privacy-Preserving Traffic Flow Prediction: A Federated Learning Approach

Can We Use Split Learning on 1D CNN Models for Privacy Preserving Training?

A Trustworthy Privacy Preserving Framework for Machine Learning in Industrial IoT Systems

Deep Learning for Financial Applications : A Survey

Split Learning for collaborative deep learning in healthcare

Advances and Open Problems in Federated Learning

One Pixel Image and RF Signal Based Split Learning for mmWave Received Power Prediction

Federated Learning With Differential Privacy: Algorithms and Performance Analysis

Federated Generative Privacy

Detailed comparison of communication efficiency of split learning and federated learning

Federated Learning: Challenges, Methods, and Future Directions

Local Differential Privacy for Deep Learning

An Efficient and Scalable Privacy Preserving Algorithm for Big Data and Data Streams

Stolen Memories: Leveraging Model Memorization for Calibrated White-Box Membership Inference

Machine Learning in Transportation

Federated Machine Learning

Split learning for health: Distributed deep learning without sharing raw patient data

LEAF: A Benchmark for Federated Settings

A generic framework for privacy preserving deep learning

Federated Learning for Mobile Keyboard Prediction

Distributed learning of deep neural network over multiple agents

How To Backdoor Federated Learning

Federated Learning with Non-IID Data

Privacy-Preserving Deep Learning via Additively Homomorphic Encryption

The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions

Differentially Private Federated Learning: A Client Level Perspective

Adaptive Laplace Mechanism: Differential Privacy Preservation in Deep Learning

Artificial intelligence in healthcare: past, present and future

SecureML: A System for Scalable Privacy-Preserving Machine Learning

A Survey on Homomorphic Encryption Schemes

Membership Inference Attacks Against Machine Learning Models

Deep Learning with Differential Privacy

Communication-Efficient Learning of Deep Networks from Decentralized Data

Deep Residual Learning for Image Recognition

Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures

Very Deep Convolutional Networks for Large-Scale Image Recognition

The Algorithmic Foundations of Differential Privacy

Extremal Mechanisms for Local Differential Privacy

ImageNet classification with deep convolutional neural networks

Differentially private data release for data mining

Fully Homomorphic Encryption over the Integers

Differential Privacy: A Survey of Results

Measuring and testing dependence by correlation of distances

Protocols for secure computations

R EDUCING LEAKAGE IN DISTRIBUTED DEEP LEARNING FOR SENSITIVE HEALTH DATA

Turning HATE Into LOVE: Homomorphic Ad Hoc Threshold Encryption for Scalable MPC

MLbase: A Distributed Machine-learning System

Privacy-Preserving Multivariate Statistical Analysis: Linear Regression and Classification

ON DATA BANKS AND PRIVACY HOMOMORPHISMS

Field of Study

Journal Information

Name

Volume