Entangled Watermarks as a Defense against Model Extraction (2020-02-27T00:00:00.000000Z)

TL;DR

Entangled Watermarking Embeddings (EWE) is introduced, which encourages the model to learn common features for classifying data that is sampled from the task distribution, but also data that encodes watermarks, which forces an adversary attempting to remove watermarks that are entangled with legitimate data to sacrifice performance on legitimate data.

Abstract

Machine learning involves expensive data collection and training procedures. Model owners may be concerned that valuable intellectual property can be leaked if adversaries mount model extraction attacks. Because it is difficult to defend against model extraction without sacrificing significant prediction accuracy, watermarking leverages unused model capacity to have the model overfit to outlier input-output pairs, which are not sampled from the task distribution and are only known to the defender. The defender then demonstrates knowledge of the input-output pairs to claim ownership of the model at inference. The effectiveness of watermarks remains limited because they are distinct from the task distribution and can thus be easily removed through compression or other forms of knowledge transfer. We introduce Entangled Watermarking Embeddings (EWE). Our approach encourages the model to learn common features for classifying data that is sampled from the task distribution, but also data that encodes watermarks. An adversary attempting to remove watermarks that are entangled with legitimate data is also forced to sacrifice performance on legitimate data. Experiments on MNIST, Fashion-MNIST, and Google Speech Commands validate that the defender can claim model ownership with 95% confidence after less than 10 queries to the stolen copy, at a modest cost of 1% accuracy in the defended model's performance.

Authors

Christopher A. Choquette-Choo

3 papers

Nicolas Papernot

3 papers

Hengrui Jia

1 papers

TL;DR

Abstract

Authors

References60 items

High Accuracy and High Fidelity Extraction of Neural Networks

High-Fidelity Extraction of Neural Network Models

Energy and Policy Considerations for Deep Learning in NLP

A framework for the extraction of Deep Neural Networks by leveraging public data

Defending Against Neural Network Model Stealing Attacks Using Deceptive Perturbations

Similarity of Neural Network Representations Revisited

Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks

Analyzing and Improving Representations with the Soft Nearest Neighbor Loss

Knockoff Nets: Stealing Functionality of Black-Box Models

Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering

Model Extraction and Active Learning

UMAP: Uniform Manifold Approximation and Projection

Model Reconstruction from Model Explanations

Copycat CNN: Stealing Knowledge by Persuading Confession with Random Non-Labeled Data

Defending Against Machine Learning Model Stealing Attacks Using Deceptive Perturbations

Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks

Protecting Intellectual Property of Deep Neural Networks with Watermarking

Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition

DeepSigns: A Generic Watermarking Framework for IP Protection of Deep Learning Models

Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learning

Adversarial Risk and the Dangers of Evaluating Against Weak Attacks

Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring

UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction

Digital watermarking for deep neural networks

Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms

BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain

Towards Deep Learning Models Resistant to Adversarial Attacks

Kapre: On-GPU Audio Preprocessing Layers for a Quick Implementation of Deep Neural Network Models with Keras

Stealing Machine Learning Models via Prediction APIs

Adversarial examples in the physical world

Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples

Practical Black-Box Attacks against Machine Learning

Practical Black-Box Attacks against Deep Learning Systems using Adversarial Examples

Deep Residual Learning for Image Recognition

The Limitations of Deep Learning in Adversarial Settings

Adam: A Method for Stochastic Optimization

Explaining and Harnessing Adversarial Examples

Adding Robustness to Support Vector Machines Against Adversarial Reverse Engineering

Principal Component Analysis (PCA)

Intriguing properties of neural networks

Vision-Based Traffic Sign Detection and Analysis for Intelligent Driver Assistance Systems: Perspectives and Survey

Algorithms for Learning Kernels Based on Centered Alignment

The German Traffic Sign Recognition Benchmark: A multi-class classification competition

A Survey on Transfer Learning

The Curse of Dimensionality

The Unreasonable Effectiveness of Data

Isolation Forest

Learning a Nonlinear Embedding by Preserving Class Neighbourhood Structure

Adversarial learning

LOF: identifying density-based local outliers

Watermarking techniques for intellectual property protection

Learning representations by back-propagating errors

New Directions in Cryptography

CSI NN: Reverse Engineering of Neural Network Architectures Through Electromagnetic Side Channel

Deep Learning

A historical perspective of speech recognition

Machine learning - a probabilistic perspective

Reading Digits in Natural Images with Unsupervised Feature Learning

Learning Multiple Layers of Features from Tiny Images

The mnist database of handwritten digits

Field of Study

Venue Information

Name

Type

URL

Alternate Names