Use of a Capsule Network to Detect Fake Images and Videos (2019-10-28T00:00:00.000000Z)

TL;DR

A capsule network that can detect various kinds of attacks, from presentation attacks using printed images and replayed videos to attacks using fake videos created using deep learning, uses many fewer parameters than traditional convolutional neural networks with similar performance.

Abstract

The revolution in computer hardware, especially in graphics processing units and tensor processing units, has enabled significant advances in computer graphics and artificial intelligence algorithms. In addition to their many beneficial applications in daily life and business, computer-generated/manipulated images and videos can be used for malicious purposes that violate security systems, privacy, and social trust. The deepfake phenomenon and its variations enable a normal user to use his or her personal computer to easily create fake videos of anybody from a short real online video. Several countermeasures have been introduced to deal with attacks using such videos. However, most of them are targeted at certain domains and are ineffective when applied to other domains or new attacks. In this paper, we introduce a capsule network that can detect various kinds of attacks, from presentation attacks using printed images and replayed videos to attacks using fake videos created using deep learning. It uses many fewer parameters than traditional convolutional neural networks with similar performance. Moreover, we explain, for the first time ever in the literature, the theory behind the application of capsule networks to the forensics problem through detailed analysis and visualization.

Authors

J. Yamagishi

12 papers

H. Nguyen

3 papers

I. Echizen

4 papers

TL;DR

Abstract

Authors

References79 items

FSGAN: Subject Agnostic Face Swapping and Reenactment

Multi-task Learning for Detecting and Segmenting Manipulated Facial Images and Videos

Detecting Photoshopped Faces by Scripting Photoshop

Text-based editing of talking-head video

Crafting A Panoptic Face Presentation Attack Detector

Deep Pixel-wise Binary Supervision for Face Presentation Attack Detection

Domain Adaptation in Multi-Channel Autoencoder based Features for Robust Face Anti-Spoofing

Vulnerability assessment and detection of Deepfake videos

Face Anti-spoofing using Hybrid Residual Learning Framework

Improving Cross-database Face Presentation Attack Detection via Adversarial Domain Adaptation

Combining Multiple one-class Classifiers for Anomaly based Face Spoofing Attack Detection

Few-Shot Adversarial Learning of Realistic Neural Talking Head Models

Recurrent Convolutional Strategies for Face Manipulation Detection in Videos

Generalized Presentation Attack Detection: a face anti-spoofing evaluation proposal

ICface: Interpretable and Controllable Face Reenactment Using GANs

RoPAD: Robust Presentation Attack Detection through Unsupervised Adversarial Invariance

FaceForensics++: Learning to Detect Manipulated Facial Images

Hybrid LSTM and Encoder–Decoder Architecture for Detection of Image Forgeries

Do GANs Leave Artificial Fingerprints?

A Style-Based Generator Architecture for Generative Adversarial Networks

ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detection

Capsule-forensics: Using Capsule Networks to Detect Forged Images and Videos

MS-CapsNet: A Novel Multi-Scale Capsule Network

MesoNet: a Compact Facial Video Forgery Detection Network

Speaker Inconsistency Detection in Tampered Video

Modular Convolutional Neural Network for Discriminating between Computer-Generated Images and Photographic Images

In Ictu Oculi: Exposing AI Generated Fake Face Videos by Detecting Eye Blinking

Deep video portraits

Learning Rich Features for Image Manipulation Detection

Distinguishing Between Natural and Computer-Generated Images Using Convolutional Neural Networks

Convolutional capsule network for classification of breast cancer histology images

Detection of GAN-Generated Fake Images over Social Networks

Investigating Capsule Networks with Dynamic Routing for Text Classification

FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces

CapsuleGAN: Generative Adversarial Capsule Network

Matrix capsules with EM routing

Spectral Capsule Networks

Capsule Network Performance on Complex Data

Distinguishing computer graphics from natural images using convolution neural networks

Recent advances in biometrie security: A case study of liveness detection in face recognition

Bringing portraits to life

Dynamic Routing Between Capsules

Exploiting Spatial Structure for Localizing Manipulated Image Regions

Demo of FaceVR: real-time facial reenactment and eye gaze control in virtual reality

Two-Stream Neural Networks for Tampered Face Detection

Transferable Deep-CNN Features for Detecting Digital and Print-Scanned Morphed Face Images

Recasting Residual-based Local Descriptors as Convolutional Neural Networks: an Application to Image Forgery Detection

Deep face liveness detection based on nonlinear diffusion using convolution neural network

FaceVR: Real-Time Facial Reenactment and Eye Gaze Control in Virtual Reality

Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization

Xception: Deep Learning with Depthwise Separable Convolutions

Face2Face: Real-Time Face Capture and Reenactment of RGB Videos

A Deep Learning Approach to Universal Image Manipulation Detection Using a New Convolutional Layer

SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation

VDub: Modifying Face Video of Actors for Plausible Visual Alignment to a Dubbed Audio Track

Face Liveness Detection From a Single Image via Diffusion Speed Model

RAISE: a raw images dataset for digital image forensics

Adam: A Method for Stochastic Optimization

Fully convolutional networks for semantic segmentation

Deep Representations for Iris, Face, and Fingerprint Spoofing Detection

Very Deep Convolutional Networks for Large-Scale Image Recognition

ImageNet Large Scale Visual Recognition Challenge

Learn Convolutional Neural Network for Face Anti-Spoofing

Handbook of Biometric Anti-Spoofing

Can face anti-spoofing countermeasures work in a real world scenario?

On the effectiveness of local binary patterns in face anti-spoofing

Rich Models for Steganalysis of Digital Images

Video face replacement

Transforming Auto-Encoders

Protecting World Leaders Against Deep Fakes

End-to-End Speech-Driven Realistic Facial Animation with Temporal GANs

“Pytorch cnn visualizations,”

“Deferred neural rendering: Image synthesis using neural textures,”

“Synthesizing obama: learning lip sync from audio,”

Trans - forming auto - encoders , ” in International Conference on Artificial Neural Networks ( ICANN )

The Digital Emily Project: Achieving a Photorealistic Digital Actor