Your Local GAN: Designing Two Dimensional Local Attention Mechanisms for Generative Models (2019-11-27T00:00:00.000000Z)

TL;DR

This work introduces a new local sparse attention layer that preserves two-dimensional geometry and locality in SAGAN and presents a novel way to invert Generative Adversarial Networks with attention.

Abstract

We introduce a new local sparse attention layer that preserves two-dimensional geometry and locality. We show that by just replacing the dense attention layer of SAGAN with our construction, we obtain very significant FID, Inception score and pure visual improvements. FID score is improved from 18.65 to 15.94 on ImageNet, keeping all other parameters the same. The sparse attention patterns that we propose for our new layer are designed using a novel information theoretic criterion that uses information flow graphs. We also present a novel way to invert Generative Adversarial Networks with attention. Our method uses the attention layer of the discriminator to create an innovative loss function. This allows us to visualize the newly introduced attention heads and show that they indeed capture interesting aspects of two-dimensional geometry of real images.

Authors

A. Dimakis

3 papers

Augustus Odena

4 papers

G. Daras

1 papers

TL;DR

Abstract

Authors

References27 items

Generative Adversarial Networks

Seeing What a GAN Cannot Generate

Adaptively Sparse Transformers

Lookahead Optimizer: k steps forward, 1 step back

Surfing: Iterative optimization over incrementally trained deep networks

SCRAM: Spatially Coherent Randomized Attention Maps

Adaptive Attention Span in Transformers

Generating Long Sequences with Sparse Transformers

GAN-Based Projector for Faster Recovery With Convergence Guarantees in Linear Inverse Problems

Transformer-XL: Attentive Language Models beyond a Fixed-Length Context

A Style-Based Generator Architecture for Generative Adversarial Networks

Large Scale GAN Training for High Fidelity Natural Image Synthesis

Self-Attention Generative Adversarial Networks

Task-Aware Compressed Sensing with Generative Adversarial Networks

Global Guarantees for Enforcing Deep Generative Priors by Empirical Risk

One Network to Solve Them All — Solving Linear Inverse Problems Using Deep Projection Models

Compressed Sensing using Generative Models

Precise Recovery of Latent Vectors from Generative Adversarial Networks

StackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks

Adversarial Feature Learning

Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

Very Deep Convolutional Networks for Large-Scale Image Recognition

ImageNet Large Scale Visual Recognition Challenge

Network Coding for Distributed Storage Systems

Network information flow

GPU Kernels for Block-Sparse Weights

Attention Is All You Need. arXiv e-prints, page arXiv:1706.03762

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names