Hierarchical Bayes Approach to Personalized Federated Unsupervised Learning (2024-02-19T00:00:00.000000Z)

TL;DR

This work develops adaptive algorithms that discover the balance between using limited local data and collaborative information, and develops a theoretical framework for personalized diffusion models, which shows the benefits of collaboration even under heterogeneity.

Abstract

Statistical heterogeneity of clients' local data is an important characteristic in federated learning, motivating personalized algorithms tailored to the local data statistics. Though there has been a plethora of algorithms proposed for personalized supervised learning, discovering the structure of local data through personalized unsupervised learning is less explored. We initiate a systematic study of such personalized unsupervised learning by developing algorithms based on optimization criteria inspired by a hierarchical Bayesian statistical framework. We develop adaptive algorithms that discover the balance between using limited local data and collaborative information. We do this in the context of two unsupervised learning tasks: personalized dimensionality reduction and personalized diffusion models. We develop convergence analyses for our adaptive algorithms which illustrate the dependence on problem parameters (e.g., heterogeneity, local sample size). We also develop a theoretical framework for personalized diffusion models, which shows the benefits of collaboration even under heterogeneity. We finally evaluate our proposed algorithms using synthetic and real data, demonstrating the effective sample amplification for personalized tasks, induced through collaboration, despite data heterogeneity.

Authors

Kaan Ozkara

1 papers

Bruce Huang

1 papers

Ruida Zhou

1 papers

TL;DR

Abstract

Authors

References46 items

Federated Learning with Diffusion Models for Privacy-Sensitive Vision Tasks

Subject-Diffusion: Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuning

Personalized PCA for Federated Heterogeneous Data

Understanding Diffusion Objectives as the ELBO with Simple Data Augmentation

Adding Conditional Control to Text-to-Image Diffusion Models

Personalized Federated Recommender Systems with Private and Partially Federated AutoEncoders

DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation

Personalized Federated Learning on Non-IID Data via Group-based Meta-learning

Personalized PCA: Decoupling Shared and Unique Features

FedPop: A Bayesian Approach for Personalised Federated Learning

Self-Aware Personalized Federated Learning

PerFED-GAN: Personalized Federated Learning via Generative Adversarial Networks

Federated Multi-Task Learning under a Mixture of Distributions

QuPeD: Quantized Personalization via Distillation with Applications to Federated Learning

Federated Variational Autoencoder for Collaborative Filtering

Decentralized Riemannian Gradient Descent on the Stiefel Manifold

Exploiting Shared Representations for Personalized Federated Learning

Score-Based Generative Modeling through Stochastic Differential Equations

Denoising Diffusion Implicit Models

An Efficient Framework for Clustered Federated Learning

Personalized Federated Learning with Moreau Envelopes

Adaptive Personalized Federated Learning

Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?

Three Approaches for Personalization with Applications to Federated Learning

Few-Shot Learning via Learning the Representation, Provably

Personalized Federated Learning: A Meta-Learning Approach

Federated Learning of a Mixture of Global and Local Models

Advances and Open Problems in Federated Learning

Adaptive Gradient-Based Meta-Learning Methods

Fully Decentralized Joint Learning of Personalized Models and Collaboration Graphs

Federated Optimization in Heterogeneous Networks

Demystifying MMD GANs

GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium

Federated Multi-Task Learning

Decentralized Collaborative Learning of Personalized Models over Networks

Communication-Efficient Learning of Deep Networks from Decentralized Data

Variational Inference with Normalizing Flows

Deep Unsupervised Learning using Nonequilibrium Thermodynamics

Large-scale inference: empirical Bayes methods for estimation, testing, and prediction

Fast Noise Variance Estimation

A Statistical Framework for Personalized Federated Learning and Estimation: Theory, Algorithms, and Privacy

FedGM: Heterogeneous Federated Learning via Generative Learning and Mutual Distillation

Fine-tuning Diffusion Models with Limited Data

Debiasing Model Updates for Improving Personalized Federated Training

On Density Estimation with Diffusion Models

Probabilistic Principal Component Analysis

Field of Study

Venue Information

Name

Type

URL

Alternate Names