Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Models (2021-03-29T00:00:00.000000Z)

TL;DR

This paper finds that it is possible to hack the model in a data-free way by modifying one single word embedding vector, with almost no accuracy sacrificed on clean samples.

Abstract

Recent studies have revealed a security threat to natural language processing (NLP) models, called the Backdoor Attack. Victim models can maintain competitive performance on clean samples while behaving abnormally on samples with a specific trigger word inserted. Previous backdoor attacking methods usually assume that attackers have a certain degree of data knowledge, either the dataset which users would use or proxy datasets for a similar task, for implementing the data poisoning procedure. However, in this paper, we find that it is possible to hack the model in a data-free way by modifying one single word embedding vector, with almost no accuracy sacrificed on clean samples. Experimental results on sentiment analysis and sentence-pair classification tasks show that our method is more efficient and stealthier. We hope this work can raise the awareness of such a critical security risk hidden in the embedding layers of NLP models. Our code is available at https://github.com/lancopku/Embedding-Poisoning.

Authors

Xu Sun

14 papers

Bin He

3 papers

Lei Li

8 papers

TL;DR

Abstract

Authors

References38 items

Red Alarm for Pre-trained Models: Universal Vulnerabilities by Neuron-Level Backdoor Attacks

Input-Aware Dynamic Backdoor Attack

Poison Attacks against Text Datasets with Conditional Adversarially Regularized Autoencoder

Can Adversarial Weight Perturbations Inject Neural Backdoors

Trojaning Language Models for Fun and Profit

Practical Detection of Trojan Neural Networks: Data-Limited and Data-Free Cases

Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks

BadNL: Backdoor Attacks Against NLP Models

Weight Poisoning Attacks on Pretrained Models

Rethinking the Trigger of Backdoor Attack

Humpty Dumpty: Controlling Word Meanings via Corpus Poisoning

NeuronInspect: Detecting Backdoors in Neural Networks via Output Explanations

Hidden Trigger Backdoor Attacks

DeepInspect: A Black-box Trojan Detection and Mitigation Framework for Deep Neural Networks

RoBERTa: A Robustly Optimized BERT Pretraining Approach

XLNet: Generalized Autoregressive Pretraining for Language Understanding

A Backdoor Attack Against LSTM-Based Text Classification Systems

Zero-Shot Knowledge Distillation in Deep Networks

Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks

Human

BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain

Attention is All you Need

Pointer Sentinel Mixture Models

SQuAD: 100,000+ Questions for Machine Comprehension of Text

Deep Residual Learning for Image Recognition

Adam: A Method for Stochastic Optimization

Sequence to Sequence Learning with Neural Networks

Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank

ImageNet classification with deep convolutional neural networks

Learning Word Vectors for Sentiment Analysis

Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification

Long Short-Term Memory

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Trojaning Attack on Neural Networks

Trojaning Attack on Neural Networks

GENERATIVE ADVERSARIAL NETS

Domain adaptation for sentiment classiﬁcation

2020 - 16th European Conference, Glasgow, UK, August 23-28

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names