Trap and Replace: Defending Backdoor Attacks by Trapping Them into an Easy-to-Replace Subnetwork (2022-10-12T00:00:00.000000Z)

TL;DR

A brand-new backdoor defense strategy, Trap and Replace, which makes it much easier to remove the harmful influence of backdoor samples from the model, and outperforms previous state-of-the-art methods.

Abstract

Deep neural networks (DNNs) are vulnerable to backdoor attacks. Previous works have shown it extremely challenging to unlearn the undesired backdoor behavior from the network, since the entire network can be affected by the backdoor samples. In this paper, we propose a brand-new backdoor defense strategy, which makes it much easier to remove the harmful influence of backdoor samples from the model. Our defense strategy, Trap and Replace, consists of two stages. In the first stage, we bait and trap the backdoors in a small and easy-to-replace subnetwork. Specifically, we add an auxiliary image reconstruction head on top of the stem network shared with a light-weighted classification head. The intuition is that the auxiliary image reconstruction task encourages the stem network to keep sufficient low-level visual features that are hard to learn but semantically correct, instead of overfitting to the easy-to-learn but semantically incorrect backdoor correlations. As a result, when trained on backdoored datasets, the backdoors are easily baited towards the unprotected classification head, since it is much more vulnerable than the shared stem, leaving the stem network hardly poisoned. In the second stage, we replace the poisoned light-weighted classification head with an untainted one, by re-training it from scratch only on a small holdout dataset with clean samples, while fixing the stem network. As a result, both the stem and the classification head in the final network are hardly affected by backdoor training samples. We evaluate our method against ten different backdoor attacks. Our method outperforms previous state-of-the-art methods by up to 20.57%, 9.80%, and 13.72% attack success rate and on-average 3.14%, 1.80%, and 1.21% clean classification accuracy on CIFAR10, GTSRB, and ImageNet-12, respectively. Code is available at https://github.com/VITA-Group/Trap-and-Replace-Backdoor-Defense.

References75 items

Board

Better Trigger Inversion Optimization in Backdoor Scanning

Quarantine: Sparsity Can Uncover the Trojan Attack Trigger for Free

Narcissus: A Practical Clean-Label Backdoor Attack with Limited Information

Backdoor Defense via Decoupling the Training Process

Few-shot Backdoor Defense Using Shapley Estimation

Towards Practical Deployment-Stage Backdoor Attack on Deep Neural Networks

AEVA: Black-box Backdoor Detection Using Adversarial Extreme Value Analysis

Adversarial Neuron Pruning Purifies Backdoored Deep Models

Anti-Backdoor Learning: Training Clean Models on Poisoned Data

Trigger Hunting with a Topological Prior for Trojan Detection

Adversarial Unlearning of Backdoors via Implicit Hypergradient

How to Inject Backdoors with Better Consistency: Logit Anchoring on Clean Data

Excess Capacity and Backdoor Poisoning

Poisoning and Backdooring Contrastive Learning

MaxUp: Lightweight Adversarial Training with Data Augmentation Improves Neural Network Training

Backdoor Attacks on Self-Supervised Learning

DeHiB: Deep Hidden Backdoor Attack on Semi-supervised Learning via Adversarial Perturbation

Poisoning the Unlabeled Dataset of Semi-Supervised Learning

Rethinking the Backdoor Attacks’ Triggers: A Frequency Perspective

Black-box Detection of Backdoor Attacks with Limited Information and Data

Targeted Attack against Deep Neural Networks via Flipping Limited Weight Bits

Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks

Invisible Backdoor Attack with Sample-Specific Triggers

Strong Data Augmentation Sanitizes Poisoning and Backdoor Attacks Without an Accuracy Tradeoff

Can Adversarial Weight Perturbations Inject Neural Backdoors

Just How Toxic is Data Poisoning? A Unified Benchmark for Backdoor and Data Poisoning Attacks

Label-Consistent Backdoor Attacks

Robust Anomaly Detection and Backdoor Attack Detection Via Differential Privacy

Latent Backdoor Attacks on Deep Neural Networks

Detecting AI Trojans Using Meta Neural Analysis

Hidden Trigger Backdoor Attacks

Invisible Backdoor Attacks on Deep Neural Networks Via Steganography and Regularization

TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systems

DeepInspect: A Black-box Trojan Detection and Mitigation Framework for Deep Neural Networks

BadNets: Evaluating Backdooring Attacks on Deep Neural Networks

Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks

STRIP: a defence against trojan attacks on deep neural networks

A New Backdoor Attack in CNNS by Training Set Corruption Without Label Poisoning

Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering

Spectral Signatures in Backdoor Attacks

Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation

Practical Fault Attack on Deep Neural Networks

Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks

Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning

mixup: Beyond Empirical Risk Minimization

Towards Deep Learning Models Resistant to Adversarial Attacks

Understanding Black-box Predictions via Influence Functions

Wide Residual Networks

Perceptual Losses for Real-Time Style Transfer and Super-Resolution

Deep Residual Learning for Image Recognition

Rethinking the Inception Architecture for Computer Vision

Intriguing properties of neural networks

Detection of traffic signs in real-world images: The German traffic sign detection benchmark

ImageNet: A large-scale hierarchical image database

Differential Privacy: A Survey of Results

Image up-sampling using total-variation regularization with a new observation model

Soroush Abbasi Koohpayegani, and Hamed Pirsiavash

What Doesn't Kill You Makes You Robust(er): Adversarial Training against Poisons and Backdoors

SPECTRE: Defending Against Backdoor Attacks Using Robust Covariance Estimation

Trojaning Attack on Neural Networks

Dropout: a simple way to prevent neural networks from overfitting

Learning Multiple Layers of Features from Tiny Images

Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content

(a) Did you state the full set of assumptions of all theoretical results

Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Have you read the ethics review guidelines and ensured that your paper conforms to them

Did you mention the license of the assets? [Yes] All datasets we used are public and cited properly

c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

c) Did you include any new assets either in the supplemental material or as a URL?

If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators? [Yes]

b) Did you describe any potential participant risks, with links to Institutional Review

Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

The defender has a small clean holdout set to sanitize the backdoored model

b) Did you mention the license of the assets? [Yes]