Ditto: Fair and Robust Federated Learning Through Personalization (2020-12-08T00:00:00.000000Z)

TL;DR

This work identifies that robustness to data and model poisoning attacks and fairness, measured as the uniformity of performance across devices, are competing constraints in statistically heterogeneous networks and proposes a simple, general framework, Ditto, that can inherently provide fairness and robustness benefits.

Abstract

Fairness and robustness are two important concerns for federated learning systems. In this work, we identify that robustness to data and model poisoning attacks and fairness, measured as the uniformity of performance across devices, are competing constraints in statistically heterogeneous networks. To address these constraints, we propose employing a simple, general framework for personalized federated learning, Ditto, that can inherently provide fairness and robustness benefits, and develop a scalable solver for it. Theoretically, we analyze the ability of Ditto to achieve fairness and robustness simultaneously on a class of linear problems. Empirically, across a suite of federated datasets, we show that Ditto not only achieves competitive performance relative to recent personalization methods, but also enables more accurate, robust, and fair models relative to state-of-the-art fair or robust baselines.

Authors

Ahmad Beirami

2 papers

Virginia Smith

6 papers

Tian Li

2 papers

TL;DR

Abstract

Authors

References70 items

Distributionally Robust Federated Averaging

Federated Reconstruction: Partially Local Federated Learning

Personalized Federated Learning with First Order Model Optimization

Towards Building a Robust and Fair Federated Learning System

Lower Bounds and Optimal Algorithms for Personalized Federated Learning

WAFFLe: Weight Anonymized Factorization for Federated Learning

Attack of the Tails: Yes, You Really Can Backdoor Federated Learning

FedFast: Going Beyond Average for Faster Training of Federated Recommender Systems

Tilted Empirical Risk Minimization

Byzantine-Robust Learning on Heterogeneous Datasets via Resampling

On Adversarial Bias and the Robustness of Fair Machine Learning

An Efficient Framework for Clustered Federated Learning

Personalized Federated Learning with Moreau Envelopes

DBA: Distributed Backdoor Attacks against Federated Learning

Data Poisoning Attacks on Federated Machine Learning

MetaPoison: Practical General-purpose Clean-label Data Poisoning

Adaptive Personalized Federated Learning

Adaptive Federated Optimization

Three Approaches for Personalization with Applications to Federated Learning

Personalized Federated Learning: A Meta-Learning Approach

Salvaging Federated Learning by Local Adaptation

Federated Learning of a Mixture of Global and Local Models

Think Locally, Act Globally: Federated Learning with Local and Global Representations

Robust Aggregation for Federated Learning

Advances and Open Problems in Federated Learning

Local Model Poisoning Attacks to Byzantine-Robust Federated Learning

Can You Really Backdoor Federated Learning?

SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization

Federated Evaluation of On-device Personalization

SCAFFOLD: Stochastic Controlled Averaging for Federated Learning

Clustered Federated Learning: Model-Agnostic Distributed Multitask Optimization Under Privacy Constraints

Improving Federated Learning Personalization via Model Agnostic Meta Learning

Differentially Private Meta-Learning

On the Convergence of FedAvg on Non-IID Data

Adaptive Gradient-Based Meta-Learning Methods

Fair Resource Allocation in Federated Learning

Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks

Agnostic Federated Learning

Federated Optimization in Heterogeneous Networks

Backdooring Convolutional Neural Networks via Targeted Weight Perturbations

LEAF: A Benchmark for Federated Settings

Analyzing Federated Learning through an Adversarial Lens

RSA: Byzantine-Robust Stochastic Aggregation Methods for Distributed Learning from Heterogeneous Datasets

How To Backdoor Federated Learning

Fairness Without Demographics in Repeated Loss Minimization

Federated Learning with Non-IID Data

Progress & Compress: A scalable framework for continual learning

Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networks

Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates

Federated Meta-Learning with Fast Convergence and Efficient Communication

Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning

Machine Learning with Adversaries: Byzantine Tolerant Gradient Descent

Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms

BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain

Federated Multi-Task Learning

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks

EMNIST: an extension of MNIST to handwritten letters

Overcoming catastrophic forgetting in neural networks

Communication-Efficient Learning of Deep Networks from Decentralized Data

Global Games With Noisy Information Sharing

Deep Learning Face Attributes in the Wild

Poisoning Attacks against Support Vector Machines

Support Vector Machines Under Adversarial Label Noise

Regularized multi--task learning

Vehicle classification in distributed sensor networks

The Byzantine Generals Problem

Federated residual Ditto: Fair and Robust Federated Learning Through Personalization learning

PAC Identiﬁability in Federated Personalization

Trojaning Attack on Neural Networks

Tensorflow federated: Machine learning on decentralized data

Field of Study

Venue Information

Name

Type

URL

Alternate Names