Eliminating Backdoor Triggers for Deep Neural Networks Using Attention Relation Graph Distillation (2022-04-21T00:00:00.000000Z)

TL;DR

A novel backdoor defense framework named Attention Relation Graph Distillation (ARGD) is introduced, which fully explores the correlation among attention features with different orders using the proposed Attentionrelation Graphs (ARGs).

Abstract

Due to the prosperity of Artificial Intelligence (AI) techniques, more and more backdoors are designed by adversaries to attack Deep Neural Networks (DNNs). Although the state-of-the-art method Neural Attention Distillation (NAD) can effectively erase backdoor triggers from DNNs, it still suffers from non-negligible Attack Success Rate (ASR) together with lowered classification ACCuracy (ACC), since NAD focuses on backdoor defense using attention features (i.e., attention maps) of the same order. In this paper, we introduce a novel backdoor defense framework named Attention Relation Graph Distillation (ARGD), which fully explores the correlation among attention features with different orders using our proposed Attention Relation Graphs (ARGs). Based on the alignment of ARGs between teacher and student models during knowledge distillation, ARGD can more effectively eradicate backdoors than NAD. Comprehensive experimental results show that, against six latest backdoor attacks, ARGD outperforms NAD by up to 94.85% reduction in ASR, while ACC can be improved by up to 3.23%.

Authors

Jun Xia

2 papers

Ting Wang

2 papers

Xian Wei

2 papers

TL;DR

Abstract

Authors

References30 items

Self-boosting for Feature Distillation

Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks

Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks

Interpreting and Disentangling Feature Components of Various Complexity from DNNs

FDA$^3$: Federated Defense Against Adversarial Attacks for Cloud-Based IIoT Applications

Backdoor Attacks Against Deep Learning Systems in the Physical World

Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustness

Rethinking the Trigger of Backdoor Attack

Pay Attention to the Activations: A Modular Attention Mechanism for Fine-Grained Image Recognition

Knowledge Distillation via Instance Relationship Graph

Adversarially Robust Distillation

BadNets: Evaluating Backdooring Attacks on Deep Neural Networks

Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks

SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systems

IEEE S&P

Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering

Spectral Signatures in Backdoor Attacks

Clean-Label Backdoor Attacks

Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation

How To Backdoor Federated Learning

Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring

Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning

Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer

Towards Evaluating the Robustness of Neural Networks

Deep Residual Learning for Image Recognition

Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks

Bilinear classifiers for visual recognition

In AAAI

Trojaning Attack on Neural Networks

In CVPR

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names