Revisiting Data-Free Knowledge Distillation with Poisoned Teachers (2023-06-04T00:00:00.000000Z)

TL;DR

This work proposes Anti-Backdoor Data-Free KD (ABD), the first plug-in defensive method for data-free KD methods to mitigate the chance of potential backdoors being transferred and empirically evaluates the effectiveness of the proposed ABD in diminishing transferred backdoor knowledge while maintaining compatible downstream performances as the vanilla KD.

Abstract

Data-free knowledge distillation (KD) helps transfer knowledge from a pre-trained model (known as the teacher model) to a smaller model (known as the student model) without access to the original training data used for training the teacher model. However, the security of the synthetic or out-of-distribution (OOD) data required in data-free KD is largely unknown and under-explored. In this work, we make the first effort to uncover the security risk of data-free KD w.r.t. untrusted pre-trained models. We then propose Anti-Backdoor Data-Free KD (ABD), the first plug-in defensive method for data-free KD methods to mitigate the chance of potential backdoors being transferred. We empirically evaluate the effectiveness of our proposed ABD in diminishing transferred backdoor knowledge while maintaining compatible downstream performances as the vanilla KD. We envision this work as a milestone for alarming and mitigating the potential backdoors in data-free KD. Codes are released at https://github.com/illidanlab/ABD.

Authors

Junyuan Hong

3 papers

Jiayu Zhou

3 papers

L. Lyu

2 papers

TL;DR

Abstract

Authors

References63 items

InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions

When FLUE Meets FLANG: Benchmarks and Large Pretrained Language Model for Financial Domain

Trap and Replace: Defending Backdoor Attacks by Trapping Them into an Easy-to-Replace Subnetwork

Resilient and Communication Efficient Learning for Heterogeneous Federated Systems

Up to 100x Faster Data-Free Knowledge Distillation

Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learning

PoisonedEncoder: Poisoning the Unlabeled Pre-training Data in Contrastive Learning

CoCa: Contrastive Captioners are Image-Text Foundation Models

Narcissus: A Practical Clean-Label Backdoor Attack with Limited Information

Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning

DENSE: Data-Free One-Shot Federated Learning

Anti-Backdoor Learning: Training Clean Models on Poisoned Data

Adversarial Unlearning of Backdoors via Implicit Hypergradient

Communication-efficient federated learning via knowledge distillation

BadEncoder: Backdoor Attacks to Pre-trained Encoders in Self-Supervised Learning

Sleeper Agent: Scalable Hidden Trigger Backdoors for Neural Networks Trained from Scratch

The Medical Segmentation Decathlon

Knowledge distillation: A good teacher is patient and consistent

Data-Free Knowledge Distillation for Heterogeneous Federated Learning

Contrastive Model Inversion for Data-Free Knowledge Distillation

MLP-Mixer: An all-MLP Architecture for Vision

SPECTRE: Defending Against Backdoor Attacks Using Robust Statistics

Rethinking the Backdoor Attacks’ Triggers: A Frequency Perspective

Towards Personalized Federated Learning

Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks

Data-free Knowledge Distillation for Object Detection

Invisible Backdoor Attack with Sample-Specific Triggers

Federated Knowledge Distillation

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition

LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention

Backdoor Learning: A Survey

On the Iteration Complexity of Hypergradient Computation

Revisiting Knowledge Distillation via Label Smoothing Regularization

Data-Free Knowledge Amalgamation via Group-Stack Dual-GAN

Entangled Watermarks as a Defense against Model Extraction

Label-Consistent Backdoor Attacks

Meta-Learning with Implicit Gradients

Invisible Backdoor Attacks on Deep Neural Networks Via Steganography and Regularization

Zero-shot Knowledge Transfer via Adversarial Belief Matching

BadNets: Evaluating Backdooring Attacks on Deep Neural Networks

Data-Free Learning of Student Networks

How to prove your model belongs to you: a blind-watermark based framework to protect intellectual property of DNN

A New Backdoor Attack in CNNS by Training Set Corruption Without Label Poisoning

Spectral Signatures in Backdoor Attacks

How To Backdoor Federated Learning

Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning

Federated Learning: Strategies for Improving Communication Efficiency

Wide Residual Networks

Distilling the Knowledge in a Neural Network

Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition

Attribute and simile classifiers for face verification

MECTA: Memory-Economic Continual Test-Time Model Adaptation

Data-Free One-Shot Federated Learning Under Very High Statistical Heterogeneity

Randomized Channel Shuffling: Minimal-Overhead Backdoor Attack Detection without Clean Datasets

An Invisible Black-Box Backdoor Attack Through Frequency Domain

Extrapolating from a Single Image to a Thousand Classes using Distillation

Wanet-imperceptible warping-based backdoor attack

Revisiting Data-Free Knowledge

Trojaning Attack on Neural Networks

Deep Face Recognition

Learning Multiple Layers of Features from Tiny Images

Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition

Field of Study

Venue Information

Name

Type

URL

Alternate Names