Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment
TL
TL;DR
TextFooler is presented, a simple but strong baseline to generate adversarial text that outperforms previous attacks by success rate and perturbation rate, and is utility-preserving and efficient, which generates adversarialtext with computational complexity linear to the text length.
Abstract
Machine learning algorithms are often vulnerable to adversarial examples that have imperceptible alterations from the original counterparts but can fool the state-of-the-art models. It is helpful to evaluate or even improve the robustness of these models by exposing the maliciously crafted adversarial examples. In this paper, we present TextFooler, a simple but strong baseline to generate adversarial text. By applying it to two fundamental natural language tasks, text classification and textual entailment, we successfully attacked three target models, including the powerful pre-trained BERT, and the widely used convolutional and recurrent neural networks. We demonstrate three advantages of this framework: (1) effective—it outperforms previous attacks by success rate and perturbation rate, (2) utility-preserving—it preserves semantic content, grammaticality, and correct types classified by humans, and (3) efficient—it generates adversarial text with computational complexity linear to the text length.1
References37 items
1
Probing Neural Network Comprehension of Natural Language Arguments
2
IMaT: Unsupervised Text Attribute Transfer via Iterative Matching and Translation
3
TextBugger: Generating Adversarial Text Against Real-world Applications
4
SALSA-TEXT : self attentive latent space based adversarial text generation
5