MeVGAN: GAN-based plugin model for video generation with applications in colonoscopy (2023-11-07T00:00:00.000000Z)

TL;DR

The Memory Efficient Video GAN (MeVGAN) is introduced–a Generative Adversarial Network (GAN) that incorporates a plugin-type architecture that utilizes a pre-trained 2D-image GAN to develop specific trajectories within the noise space.

Abstract

The generation of videos is crucial, particularly in the medical field, where a significant amount of data is presented in this format. However, due to the extensive memory requirements, creating high-resolution videos poses a substantial challenge for generative models. In this paper, we introduce the Memory Efficient Video GAN (MeVGAN)–a Generative Adversarial Network (GAN) that incorporates a plugin-type architecture. This system utilizes a pre-trained 2D-image GAN, to which we attach a straightforward neural network designed to develop specific trajectories within the noise space. These trajectories, when processed through the GAN, produce realistic videos. We deploy MeVGAN specifically for creating colonoscopy videos, a critical procedure in the medical field, notably helpful for screening and treating colorectal cancer. We show that MeVGAN can produce good quality synthetic colonoscopy videos, which can be potentially used in virtual simulators.

Authors

P. Spurek

2 papers

Lukasz Struski

1 papers

Tomasz Urba'nczyk

1 papers

TL;DR

Abstract

Authors

References70 items

Synthetic data in health care: A narrative review

Encoder Enabled Gan-based Video Generators

Polyp detection on video colonoscopy using a hybrid 2D/3D CNN

GAN Inversion for Data Augmentation to Improve Colonoscopy Lesion Classification

Colonoscopy: Preparation and Potential Complications

Generating Videos with Dynamics-aware Implicit Generative Adversarial Networks

StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2

PluGeN: Multi-Label Conditional Generation From Pre-Trained Models

Dynamic GAN for high-quality sign language video generation from skeletal poses using generative adversarial networks

StyleVideoGAN: A Temporal Generative Model using a Pretrained StyleGAN

Cervical Cancer Diagnostics Healthcare System Using Hybrid Object Detection Adversarial Networks

SinGAN-Seg: Synthetic training data generation for medical image segmentation

A Good Image Generator Is What You Need for High-Resolution Video Synthesis

Efficient texture-aware multi-GAN for image inpainting

Imaging based cervical cancer diagnostics using small object detection - generative adversarial networks

GAN-Based Synthetic Gastrointestinal Image Generation

Video Generative Adversarial Networks: A Review

Training Generative Adversarial Networks with Limited Data

Temporal Shift GAN for Large Scale Video Generation

Lower Dimensional Kernels for Video Discriminators

G3AN: Disentangling Appearance and Motion for Video Generation

Adversarial Video Generation on Complex Datasets

Endoscopic simulators.

SinGAN: Learning a Generative Model From a Single Natural Image

FVD: A new Metric for Video Generation

Semantic Image Synthesis With Spatially-Adaptive Normalization

A Style-Based Generator Architecture for Generative Adversarial Networks

TwoStreamVAN: Improving Motion Modeling in Video Generation

Train Sparsely, Generate Densely: Memory-Efficient Unsupervised Training of High-Resolution Temporal GAN

Towards High Resolution Video Generation with Progressive Growing of Sliced Wasserstein GANs

Large Scale GAN Training for High Fidelity Natural Image Synthesis

Glow: Generative Flow with Invertible 1x1 Convolutions

Self-Attention Generative Adversarial Networks

Spectral Normalization for Generative Adversarial Networks

On Convergence and Stability of GANs

Colonoscopy procedure simulation: virtual reality training based on a real time computational approach

Inferring Semantic Layout for Hierarchical Text-to-Image Synthesis

Hierarchical Video Generation from Orthogonal Information: Optical Flow and Texture

Wasserstein Auto-Encoders

Progressive Growing of GANs for Improved Quality, Stability, and Variation

MoCoGAN: Decomposing Motion and Content for Video Generation

Wasserstein Generative Adversarial Networks

GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium

Improved Training of Wasserstein GANs

Temporal Generative Adversarial Nets with Singular Value Clipping

Image-to-Image Translation with Conditional Adversarial Networks

Unsupervised Cross-Domain Image Generation

Amortised MAP Inference for Image Super-resolution

Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network

Generating Videos with Scene Dynamics

Improved Techniques for Training GANs

Generative Adversarial Text to Image Synthesis

Fellowship Colonoscopy Training and Preparedness for Independent Gastroenterology Practice

NICE: Non-linear Independent Components Estimation

The Role of Colonoscopy in Inflammatory Bowel Disease

UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild

Data Curation

Expert and construct validity of the Simbionix GI Mentor II endoscopy simulator for colonoscopy

A Survey of Haptic Rendering Techniques

Training with the compactEASIE biologic endoscopy simulator significantly improves hemostatic technical skill of gastroenterology fellows: a randomized controlled comparison with clinical endoscopy training alone.

Practical Endoscopy Training Using a New Gastrointestinal Phantom

Convolutional Two-Stream Generative Adversarial Network-Based Hyperspectral Feature Extraction

Development of an End-to-End Deep Learning Framework for Sign Language Recognition, Translation, and Video Generation

Formal Analysis

AUTO-ENCODING VARIATIONAL BAYES

GENERATIVE ADVERSARIAL NETS

Effective colonoscopy training techniques: strate- gies to improve patient outcomes

Complications of colonoscopy.

High Fidelity Haptic Rendering

The role of colonoscopy in inflammatory bowel disease.

Field of Study

Journal Information

Name

Volume

Venue Information

Name