RF-Next: Efficient Receptive Field Search for Convolutional Neural Networks (2022-06-14T00:00:00.000000Z)

TL;DR

This work proposes to find better receptive field combinations through a global-to-local search scheme, and proposes an expectation-guided iterative local search scheme to refine combinations effectively.

Abstract

Temporal/spatial receptive fields of models play an important role in sequential/spatial tasks. Large receptive fields facilitate long-term relations, while small receptive fields help to capture the local details. Existing methods construct models with hand-designed receptive fields in layers. Can we effectively search for receptive field combinations to replace hand-designed patterns? To answer this question, we propose to find better receptive field combinations through a global-to-local search scheme. Our search scheme exploits both global search to find the coarse combinations and local search to get the refined receptive field combinations further. The global search finds possible coarse combinations other than human-designed patterns. On top of the global search, we propose an expectation-guided iterative local search scheme to refine combinations effectively. Our RF-Next models, plugging receptive field search to various models, boost the performance on many tasks, e.g., temporal action segmentation, object detection, instance segmentation, and speech synthesis. The source code is publicly available on http://mmcheng.net/rfnext.

Authors

Ming-Ming Cheng

4 papers

Zhong-Yu Li

3 papers

Shanghua Gao

4 papers

TL;DR

Abstract

Authors

References154 items

Deep Continuous Networks

A ConvNet for the 2020s

A Highly Efficient Model to Study the Semantics of Salient Object Detection

A Survey on Neural Speech Synthesis

PVT v2: Improved baselines with Pyramid Vision Transformer

Large-Scale Unsupervised Semantic Segmentation

Representative Batch Normalization with Feature Calibration

Deeply Shape-guided Cascade for Instance Segmentation

A2-FPN: Attention Aggregation based Feature Pyramid Network for Instance Segmentation

A Survey of Modern Deep Learning based Object Detection Models

Look Closer to Segment Better: Boundary Patch Refinement for Instance Segmentation

OPANAS: One-Shot Path Aggregation Network Architecture Search for Object Detection

Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions

Global2Local: Efficient Structure Search for Video Action Segmentation

End-to-End Object Detection with Fully Convolutional Network

Sparse R-CNN: End-to-End Object Detection with Learnable Proposals

DCT-Mask: Discrete Cosine Transform Mask Representation for Instance Segmentation

Jittor: a novel deep learning framework with meta-operators and unified graph execution

Boundary-preserving Mask R-CNN

SegFix: Model-Agnostic Boundary Refinement for Segmentation

DetectoRS: Detecting Objects with Recursive Feature Pyramid and Switchable Atrous Convolution

VecRoad: Point-Based Iterative Graph Exploration for Road Graphs Extraction

Improving Action Segmentation via Graph-Based Temporal Reasoning

MobileDets: Searching for Object Detection Architectures for Mobile Accelerators

FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensions

Pyramid Constrained Self-Attention Network for Fast Video Salient Object Detection

SCT: Set Constrained Temporal Transformer for Set Supervised Action Segmentation

DCNAS: Densely Connected Neural Architecture Search for Semantic Image Segmentation

Highly Efficient Salient Object Detection with 100K Parameters

Deep Hough Transform for Semantic Line Detection

Action Segmentation With Joint Self-Supervised Temporal Domain Adaptation

Set-Constrained Viterbi for Set-Supervised Action Segmentation

PolyTransform: Deep Polygon Transformer for Instance Segmentation

WaveFlow: A Compact Flow-based Model for Raw Audio

PyTorch: An Imperative Style, High-Performance Deep Learning Library

Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Search

Understanding and Robustifying Differentiable Architecture Search

DARTS+: Improved Differentiable Architecture Search with Early Stopping

Deep High-Resolution Representation Learning for Visual Recognition

PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search

Cascade R-CNN: High Quality Object Detection and Instance Segmentation

Probabilistic Forecasting with Temporal Convolutional Neural Network

A Hybrid RNN-HMM Approach for Weakly Supervised Temporal Action Segmentation

Object Detection in 20 Years: A Survey

Searching for MobileNetV3

Progressive Differentiable Architecture Search: Bridging the Depth Gap Between Search and Evaluation

Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks With Octave Convolution

Res2Net: A New Multi-Scale Backbone Architecture

MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation

Deep High-Resolution Representation Learning for Human Pose Estimation

Hybrid Task Cascade for Instance Segmentation

Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation

Coupled Generative Adversarial Network for Continuous Fine-Grained Action Segmentation

SlowFast Networks for Video Recognition

Graph-Based Global Reasoning Networks

CCNet: Criss-Cross Attention for Semantic Segmentation

FloWaveNet : A Generative Flow for Raw Audio

Waveglow: A Flow-based Generative Network for Speech Synthesis

NSGA-Net: neural architecture search using multi-objective genetic algorithm

ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware

Neural Speech Synthesis with Transformer Network

Deep Learning for Generic Object Detection: A Survey

Automatically Designing CNN Architectures Using the Genetic Algorithm for Image Classification

CornerNet: Detecting Objects as Paired Keypoints

Glow: Generative Flow with Invertible 1x1 Convolutions

DARTS: Differentiable Architecture Search

Temporal Deformable Residual Networks for Action Segmentation in Videos

DenseASPP for Semantic Segmentation in Street Scenes

Large Receptive Field Networks for High-Scale Image Super-Resolution

Weakly-Supervised Action Segmentation with Iterative Soft Boundary Assignment

Unsupervised Learning and Segmentation of Complex Activities from Video

An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling

Regularized Evolution for Image Classifier Architecture Search

Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions

Non-local Neural Networks

Hierarchical Representations for Efficient Architecture Search