Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation (2020-12-13T00:00:00.000000Z)

TL;DR

A systematic study of the Copy-Paste augmentation for instance segmentation where the authors randomly paste objects onto an image finds that the simple mechanism of pasting objects randomly is good enough and can provide solid gains on top of strong baselines.

Abstract

Building instance segmentation models that are data-efficient and can handle rare object categories is an important challenge in computer vision. Leveraging data augmentations is a promising direction towards addressing this challenge. Here, we perform a systematic study of the Copy-Paste augmentation (e.g., [13], [12]) for instance segmentation where we randomly paste objects onto an image. Prior studies on Copy-Paste relied on modeling the surrounding visual context for pasting the objects. However, we find that the simple mechanism of pasting objects randomly is good enough and can provide solid gains on top of strong baselines. Furthermore, we show Copy-Paste is additive with semi-supervised methods that leverage extra data through pseudo labeling (e.g. self-training). On COCO instance segmentation, we achieve 49.1 mask AP and 57.3 box AP, an improvement of +0.6 mask AP and +1.5 box AP over the previous state-of-the-art. We further demonstrate that Copy-Paste can lead to significant improvements on the LVIS benchmark. Our baseline model outperforms the LVIS 2020 Challenge winning entry by +3.6 mask AP on rare categories.1

Authors

Quoc V. Le

42 papers

E. D. Cubuk

9 papers

A. Srinivas

6 papers

TL;DR

Abstract

Authors

References75 items

Scaled-YOLOv4: Scaling Cross Stage Partial Network

Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect

1st Place Solution of LVIS Challenge 2020: A Good Box is not a Guarantee of a Good Mask

Balanced Activation for Long-tailed Visual Recognition

The Devil is in Classification: A Simple Framework for Long-tail Object Detection and Instance Segmentation

Rethinking Pre-training and Self-training

DetectoRS: Detecting Objects with Recursive Feature Pyramid and Switchable Atrous Convolution

Overcoming Classifier Imbalance for Long-Tail Object Detection With Balanced Group Softmax

YOLOv4: Optimal Speed and Accuracy of Object Detection

ResNeSt: Split-Attention Networks

Learning to Segment the Tail

Rethinking Class-Balanced Methods for Long-Tailed Visual Recognition From a Domain Adaptation Perspective

Equalization Loss for Long-Tailed Object Recognition

A Simple Framework for Contrastive Learning of Visual Representations

SpineNet: Learning Scale-Permuted Backbone for Recognition and Localization

BBN: Bilateral-Branch Network With Cumulative Learning for Long-Tailed Visual Recognition

EfficientDet: Scalable and Efficient Object Detection

Momentum Contrast for Unsupervised Visual Representation Learning

Self-Training With Noisy Student Improves ImageNet Classification

Decoupling Representation and Classifier for Long-Tailed Recognition

Objects365: A Large-Scale, High-Quality Dataset for Object Detection

Meta-Learning to Detect Rare Objects

Randaugment: Practical automated data augmentation with a reduced search space

Imbalance Problems in Object Detection: A Review

InstaBoost: Boosting Instance Segmentation via Probability Map Guided Copy-Pasting

A survey on Image Data Augmentation for Deep Learning

Learning Data Augmentation Strategies for Object Detection

Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss

LVIS: A Dataset for Large Vocabulary Instance Segmentation

EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

Data-Efficient Image Recognition with Contrastive Predictive Coding

CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features

NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection

Bag of Freebies for Training Object Detection Neural Networks

Striking the Right Balance With Uncertainty

Class-Balanced Loss Based on Effective Number of Samples

Rethinking ImageNet Pre-Training

Modeling Visual Context is Key to Augmenting Object Detection Datasets

Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learning

AutoAugment: Learning Augmentation Policies from Data

Exploring the Limits of Weakly Supervised Pretraining

SNIPER: Efficient Multi-Scale Training

ExFuse: Enhancing Feature Fusion for Semantic Segmentation

Learning to Segment via Cut-and-Paste

Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation

Data Distillation: Towards Omni-Supervised Learning

Cascade R-CNN: Delving Into High Quality Object Detection

Single-Shot Refinement Neural Network for Object Detection

mixup: Beyond Empirical Risk Minimization

Focal Loss for Dense Object Detection

Cut, Paste and Learn: Surprisingly Easy Synthesis for Instance Detection

The iNaturalist Species Classification and Detection Dataset

Mask R-CNN

Feature Pyramid Networks for Object Detection

Multiview RGB-D Dataset for Object Instance Detection

Learning Deep Representation for Imbalanced Classification

Instance-Aware Semantic Segmentation via Multi-task Network Cascades

Deep Residual Learning for Image Recognition

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

Fast R-CNN

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Hypercolumns for object segmentation and fine-grained localization

Going deeper with convolutions

Very Deep Convolutional Networks for Large-Scale Image Recognition

ImageNet Large Scale Visual Recognition Challenge

Simultaneous Detection and Segmentation

Microsoft COCO: Common Objects in Context

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation

Regularization of Neural Networks using DropConnect

ImageNet classification with deep convolutional neural networks

Baris Can Cam, Sinan Kalkan, and Emre Akbas. Imbalance problems in object detection: A review. TPAMI

Rethinking Class-Balanced Methods for Long-Tailed Visual Recognition from a Domain Adaptation Perspective

The iNaturalist Species Classification and Detection Dataset-Supplementary Material

Gradient-based learning applied to document recognition

Edinburgh Research Explorer The PASCAL Visual Object Classes (VOC) Challenge

Field of Study