Local Patch AutoAugment With Multi-Agent Collaboration (2021-03-20T00:00:00.000000Z)

TL;DR

This paper proposes a more fine-grained automated DA approach, dubbed Patch AutoAugment, to divide an image into a grid of patches and search for the joint optimal augmentation policies for the patches as a multi-agent reinforcement learning (MARL) problem.

Abstract

Data augmentation (DA) plays a critical role in improving the generalization of deep learning models. Recent works on automatically searching for DA policies from data have achieved great success. However, existing automated DA methods generally perform the search at the image level, which limits the exploration of diversity in local regions. In this paper, we propose a more fine-grained automated DA approach, dubbed Patch AutoAugment, to divide an image into a grid of patches and search for the joint optimal augmentation policies for the patches. We formulate it as a multi-agent reinforcement learning (MARL) problem, where each agent learns an augmentation policy for each patch based on its content together with the semantics of the whole image. The agents cooperate with each other to achieve the optimal augmentation effect of the entire image by sharing a team reward. We show the effectiveness of our method on multiple benchmark datasets of image classification, fine-grained image recognition and object detection (e.g., CIFAR-10, CIFAR-100, ImageNet, CUB-200-2011, Stanford Cars, FGVC-Aircraft and Pascal VOC 2007). Extensive experiments demonstrate that our method outperforms the state-of-the-art DA methods while requiring fewer computational resources.

Authors

Xin Jin

2 papers

Shiqi Lin

1 papers

Tao Yu

1 papers

TL;DR

Abstract

Authors

References83 items

Distribution-Preserving-Based Automatic Data Augmentation for Deep Image Steganalysis

A Reinforcement-Learning-Based Energy-Efficient Framework for Multi-Task Video Analytics Pipeline

Direct Differentiable Augmentation Search

TrivialAugment: Tuning-free Yet State-of-the-Art Data Augmentation

Co-Mixup: Saliency Guided Joint Mixup with Supermodular Diversity

Boundary-Aware Supervoxel-Level Iteratively Refined Interactive 3D Image Segmentation With Multi-Agent Reinforcement Learning

Training data-efficient image transformers & distillation through attention

SnapMix: Semantically Proportional Mixing for Augmenting Fine-grained Data

Understanding Failures of Deep Networks via Robust Feature Extraction

KeepAugment: A Simple Information-Preserving Data Augmentation Approach

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks

SaliencyMix: A Saliency Guided Data Augmentation Strategy for Better Regularization

SuperMix: Supervising the Mixing Data Augmentation

Fine-Grained Visual Classification via Progressive Multi-Granularity Training of Jigsaw Patches

DADA: Differentiable Automatic Data Augmentation

Affinity and Diversity: Quantifying Mechanisms of Data Augmentation

Adversarial AutoAugment

PixelRL: Fully Convolutional Network With Reinforcement Learning for Image Processing

Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms

Iteratively-Refined Interactive 3D Medical Image Segmentation With Multi-Agent Reinforcement Learning

Faster AutoAugment: Learning Augmentation Strategies using Backpropagation

Kornia: an Open Source Differentiable Computer Vision Library for PyTorch

Randaugment: Practical automated data augmentation with a reduced search space

A survey on Image Data Augmentation for Deep Learning

Destruction and Construction Learning for Fine-Grained Image Recognition

Grid-Wise Control for Multi-Agent Reinforcement Learning in Video Game AI

Online Hyper-Parameter Learning for Auto-Augmentation Strategy

Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules

CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features

Fast AutoAugment

End-to-End Automatic Image Annotation Based on Deep CNN and Multi-Label Data Augmentation

SECOND: Sparsely Embedded Convolutional Detection

AutoAugment: Learning Augmentation Policies from Data

QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning

ShakeDrop regularization

A Bayesian Data Augmentation Approach for Learning Deep Models

mixup: Beyond Empirical Risk Minimization

Emergent Complexity via Multi-Agent Competition

Learning with Opponent-Learning Awareness

Random Erasing Data Augmentation

Improved Regularization of Convolutional Neural Networks with Cutout

Learning Transferable Architectures for Scalable Image Recognition

Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments

Counterfactual Multi-Agent Policy Gradients

Shake-Shake regularization

Learning from Simulated and Unsupervised Images through Adversarial Training

Large-scale multi-agent reinforcement learning using image-based state representation

Aggregated Residual Transformations for Deep Neural Networks

Deep Pyramidal Residual Networks

Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization

Wide Residual Networks

Asynchronous Methods for Deep Reinforcement Learning

Learning Deep Features for Discriminative Localization

Deep Residual Learning for Image Recognition

Multiagent cooperation and competition with deep reinforcement learning

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

Salient Region Detection via Integrating Diffusion-Based Compactness and Local Contrast

3D Object Representations for Fine-Grained Categorization

Fine-Grained Visual Classification of Aircraft

Combining Policy Search with Planning in Multi-agent Cooperation

ImageNet: A large-scale hierarchical image database

A Comprehensive Survey of Multiagent Reinforcement Learning

Planning, Learning and Coordination in Multiagent Decision Processes

AutoMA: Towards Automatic Model Augmentation for Transferable Adversarial Attacks

Accurate Scene Text Detection Via Scale-Aware Data Augmentation and Shape Similarity Constraint

Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

3D Pose Estimation Based on Reinforce Learning for 2D Image-Based 3D Model Retrieval

“Patch-levelaugmentationforobjectdetec-tioninaerialimages,”in

“Augmix:Asimpledataprocessingmethodtoimprove robustnessanduncertainty,

Triantafyllos Afouras, Nantas Nardelli, and Shimon Whiteson

“TheCaltech-UCSDbirds-200-2011dataset,”CaliforniaInst

The caltech-ucsd birds

We do not specifically tune these hyperparameters, and all of these are consistent with previous works

Learning Multiple Layers of Features from Tiny Images

From Single-Agent to Multi-Agent Reinforcement Learning: Foundational Concepts and Methods