Latent Neural Differential Equations for Video Generation (2020-11-07T00:00:00.000000Z)

TL;DR

This work proposes studying the effects of Neural Differential Equations to model the temporal dynamics of video generation, and investigates how changes in temporal models affect generated video quality.

Abstract

Generative Adversarial Networks have recently shown promise for video generation, building off of the success of image generation while also addressing a new challenge: time. Although time was analyzed in some early work, the literature has not adequately grown with temporal modeling developments. We propose studying the effects of Neural Differential Equations to model the temporal dynamics of video generation. The paradigm of Neural Differential Equations presents many theoretical strengths including the first continuous representation of time within video generation. In order to address the effects of Neural Differential Equations, we will investigate how changes in temporal models affect generated video quality.

Authors

Cade Gordon

3 papers

Natalie Parde

1 papers

TL;DR

Abstract

Authors

References45 items

Generative Adversarial Networks

Moflowgan: Video Generation With Flow Guidance

Transformation-based Adversarial Video Prediction on Large-Scale Data

Stochastic Latent Residual Video Prediction

Scalable Gradients for Stochastic Differential Equations

Lower Dimensional Kernels for Video Discriminators

Analyzing and Improving the Image Quality of StyleGAN

High Fidelity Video Prediction with Large Stochastic Recurrent Neural Networks

Adversarial Video Generation on Complex Datasets

ODE$^2$VAE: Deep generative second order ODEs with Bayesian neural networks

Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limit

A Style-Based Generator Architecture for Generative Adversarial Networks

How Does Lipschitz Regularization Influence GAN Training?

Train Sparsely, Generate Densely: Memory-Efficient Unsupervised Training of High-Resolution Temporal GAN

Large Scale GAN Training for High Fidelity Natural Image Synthesis

FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models

Flow-Grounded Spatial-Temporal Video Prediction from Still Images

Neural Ordinary Differential Equations

Learning to Decompose and Disentangle Representations for Video Prediction

Controllable Video Generation with Sparse Trajectories

Stochastic Adversarial Video Prediction

Stochastic Video Generation with a Learned Prior

Spectral Normalization for Generative Adversarial Networks

Stochastic Variational Video Prediction

Progressive Growing of GANs for Improved Quality, Stability, and Variation

ContextVP: Fully Context-Aware Video Prediction

Dual Motion GAN for Future-Flow Embedded Video Prediction

MoCoGAN: Decomposing Motion and Content for Video Generation

Wasserstein Generative Adversarial Networks

GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium

Improved Training of Wasserstein GANs

Video Frame Synthesis Using Deep Voxel Flow

Temporal Generative Adversarial Nets with Singular Value Clipping

Generating Videos with Scene Dynamics

Improved Techniques for Training GANs

Unsupervised Learning for Physical Interaction through Video Prediction

Deep Residual Learning for Image Recognition

Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting

Unsupervised Learning of Video Representations using LSTMs

Video (language) modeling: a baseline for generative models of natural videos

Learning Spatiotemporal Features with 3D Convolutional Networks

Large-Scale Video Classification with Convolutional Neural Networks

UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild

Long Short-Term Memory

Field of Study