Personalized Federated Learning with Feature Alignment and Classifier Collaboration (2023-06-20T00:00:00.000000Z)

TL;DR

This work conducts explicit local-global feature alignment by leveraging global semantic knowledge for learning a better representation for each client as a function of the combining weights and derive an optimization problem for estimating optimal weights.

Abstract

Data heterogeneity is one of the most challenging issues in federated learning, which motivates a variety of approaches to learn personalized models for participating clients. One such approach in deep neural networks based tasks is employing a shared feature representation and learning a customized classifier head for each client. However, previous works do not utilize the global knowledge during local representation learning and also neglect the fine-grained collaboration between local classifier heads, which limit the model generalization ability. In this work, we conduct explicit local-global feature alignment by leveraging global semantic knowledge for learning a better representation. Moreover, we quantify the benefit of classifier combination for each client as a function of the combining weights and derive an optimization problem for estimating optimal weights. Finally, extensive evaluation results on benchmark datasets with various heterogeneous data scenarios demonstrate the effectiveness of our proposed method. Code is available at https://github.com/JianXu95/FedPAC

Authors

Xin-Yi Tong

2 papers

Jian Xu

1 papers

Shao-Lun Huang

1 papers

TL;DR

Abstract

Authors

References72 items

FedFA: Federated Learning With Feature Anchors to Align Features and Classifiers for Heterogeneous Data

FedFM: Anchor-Based Feature Matching for Data Heterogeneity in Federated Learning

Personalized Federated Learning through Local Memorization

Federated Learning Based on Dynamic Regularization

Parameterized Knowledge Transfer for Personalized Federated Learning

WAFFLE: Weighted Averaging for Personalized Federated Learning

FedProc: Prototypical Contrastive Federated Learning on Non-IID data

Personalized Federated Learning for Heterogeneous Clients with Clustered Knowledge Transfer

Federated Multi-Task Learning under a Mixture of Distributions

Flexible Clustered Federated Learning for Client-Level Data Distribution Shift

On Bridging Generic and Personalized Federated Learning

FedMix: Approximation of Mixup under Mean Augmented Federated Learning

Personalized Federated Learning with Gaussian Processes

Implicit Gradient Alignment in Distributed and Federated Learning

Federated Learning on Non-IID Data: A Survey

FedBABU: Towards Enhanced Representation for Federated Image Classification

Data-Free Knowledge Distillation for Heterogeneous Federated Learning

Prototype Guided Federated Learning of Visual Feature Representations

Node Selection Toward Faster Convergence for Federated Learning on Non-IID Data

Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learning

Model-Contrastive Federated Learning

Personalized Federated Learning using Hypernetworks

Towards Personalized Federated Learning

Exploiting Shared Representations for Personalized Federated Learning

Bias-Variance Reduced Local SGD for Less Heterogeneous Federated Learning

Federated Learning on Non-IID Data Silos: An Experimental Study

Personalized Federated Learning with First Order Model Optimization

Ditto: Fair and Robust Federated Learning Through Personalization

Addressing Class Imbalance in Federated Learning

Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning

FedML: A Research Library and Benchmark for Federated Machine Learning

Personalized Cross-Silo Federated Learning on Non-IID Data

Optimizing Federated Learning on Non-IID Data with Reinforcement Learning

Ensemble Distillation for Robust Model Fusion in Federated Learning

An Efficient Framework for Clustered Federated Learning

Personalized Federated Learning with Moreau Envelopes

Prototypical Contrastive Learning of Unsupervised Representations

Adaptive Personalized Federated Learning

Survey of Personalization Techniques for Federated Learning

Federated Visual Classification with Real-World Data Distribution

Three Approaches for Personalization with Applications to Federated Learning

Federated Learning of a Mixture of Global and Local Models

Maximal Correlation Regression

Think Locally, Act Globally: Federated Learning with Local and Global Representations

Advances and Open Problems in Federated Learning

Federated Learning with Personalization Layers

SCAFFOLD: Stochastic Controlled Averaging for Federated Learning

Clustered Federated Learning: Model-Agnostic Distributed Multitask Optimization Under Privacy Constraints

Improving Federated Learning Personalization via Model Agnostic Meta Learning

Federated Learning: Challenges, Methods, and Future Directions

Transferrable Prototypical Networks for Unsupervised Domain Adaptation

Federated Optimization in Heterogeneous Networks

Federated Learning with Non-IID Data

Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms

Federated Multi-Task Learning

EMNIST: Extending MNIST to handwritten letters

Prototypical Networks for Few-shot Learning

Communication-Efficient Learning of Deep Networks from Decentralized Data

Empirical Evaluation of Rectified Activations in Convolutional Network

ImageNet classification with deep convolutional neural networks

Deep Neural Networks for Acoustic Modeling in Speech Recognition

Representation Learning: A Review and New Perspectives

Multitask Learning: A Knowledge-Based Source of Inductive Bias

On Bridging Generic and Personalized Federated Learning for Image Classification

FedProto: Federated Prototype Learning over Heterogeneous Devices

A Mathematical Framework for Quantifying Transferability in Multi-source Transfer Learning

Debiasing Model Updates for Improving Personalized Federated Training

FedGP: Correlation-Based Active Client Selection for Heterogeneous Federated Learning

Personalized Federated Learning with Theoretical Guarantees: A Model-Agnostic Meta-Learning Approach

Deep Learning

Learning Multiple Layers of Features from Tiny Images

The method of types [information theory]

Field of Study

Journal Information

Name

Volume