Three Approaches for Personalization with Applications to Federated Learning (2020-02-25T00:00:00.000000Z)

TL;DR

This work presents a systematic learning-theoretic study of personalization, and proposes and analyzes three approaches: user clustering, data interpolation, and model interpolation.

Abstract

The standard objective in machine learning is to train a single model for all users. However, in many learning scenarios, such as cloud computing and federated learning, it is possible to learn a personalized model per user. In this work, we present a systematic learning-theoretic study of personalization. We propose and analyze three approaches: user clustering, data interpolation, and model interpolation. For all three approaches, we provide learning-theoretic guarantees and efficient algorithms for which we also demonstrate the performance empirically. All of our algorithms are model-agnostic and work for any hypothesis class.

Authors

Y. Mansour

1 papers

M. Mohri

1 papers

Jae Ro

1 papers

TL;DR

Abstract

Authors

References55 items

Foundations of Machine Learning

Adaptive Personalized Federated Learning

Federated Residual Learning

Survey of Personalization Techniques for Federated Learning

Personalized Federated Learning: A Meta-Learning Approach

Salvaging Federated Learning by Local Adaptation

Federated Learning of a Mixture of Global and Local Models

Think Locally, Act Globally: Federated Learning with Local and Global Representations

Private Federated Learning with Domain Adaptation

Advances and Open Problems in Federated Learning

Federated Learning with Personalization Layers

Generative Models for Effective ML on Private, Decentralized Datasets

Federated Evaluation of On-device Personalization

SCAFFOLD: Stochastic Controlled Averaging for Federated Learning

SCAFFOLD: Stochastic Controlled Averaging for On-Device Federated Learning

Federated Learning of N-Gram Language Models

Clustered Federated Learning: Model-Agnostic Distributed Multitask Optimization Under Privacy Constraints

Improving Federated Learning Personalization via Model Agnostic Meta Learning

Federated User Representation Learning

Federated Learning: Challenges, Methods, and Future Directions

Variational Federated Multi-Task Learning

Federated Learning for Emoji Prediction in a Mobile Keyboard

Adaptive Gradient-Based Meta-Learning Methods

Federated Learning Of Out-Of-Vocabulary Words

Agnostic Federated Learning

Federated Machine Learning

APPLIED FEDERATED LEARNING: IMPROVING GOOGLE KEYBOARD QUERY SUGGESTIONS

LEAF: A Benchmark for Federated Settings

Federated Learning for Mobile Keyboard Prediction

Federated Learning with Non-IID Data

cpSGD: Communication-efficient and differentially-private distributed SGD

Graph Oracle Models, Lower Bounds, and Gaps for Parallel Stochastic Optimization

Local SGD Converges Fast and Communicates Little

Federated Learning for Ultra-Reliable Low-Latency V2V Communications

Federated learning of predictive models from federated Electronic Health Records

Federated Meta-Learning with Fast Convergence and Efficient Communication

Practical Secure Aggregation for Privacy-Preserving Machine Learning

Recent progresses in deep learning based acoustic models

Federated Multi-Task Learning

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks

Distributed Mean Estimation with Limited Communication

Federated Learning: Strategies for Improving Communication Efficiency

Federated Optimization: Distributed Machine Learning for On-Device Intelligence

Communication-Efficient Learning of Deep Networks from Decentralized Data

Domain-Adversarial Training of Neural Networks

An Empirical Investigation of Catastrophic Forgeting in Gradient-Based Neural Networks

New Analysis and Algorithm for Learning with Drifting Distributions

Domain Adaptation: Learning Bounds and Algorithms

Domain Adaptation with Multiple Sources

Data clustering: 50 years beyond K-means

Learning Bounds for Domain Adaptation

Multilevel (Hierarchical) Modeling: What It Can and Cannot Do

Clustering with Bregman Divergences

Principles of Risk Minimization for Learning Theory

Adversarial Multiple Source Domain Adaptation

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names