RTMDet: An Empirical Study of Designing Real-Time Object Detectors (2022-12-14T00:00:00.000000Z)

TL;DR

An efficient real-time object detector is designed that exceeds the YOLO series and is easily extensible for many object recognition tasks such as instance segmentation and rotated object detection and introduces soft labels when calculating matching costs in the dynamic label assignment to improve accuracy.

Abstract

In this paper, we aim to design an efficient real-time object detector that exceeds the YOLO series and is easily extensible for many object recognition tasks such as instance segmentation and rotated object detection. To obtain a more efficient model architecture, we explore an architecture that has compatible capacities in the backbone and neck, constructed by a basic building block that consists of large-kernel depth-wise convolutions. We further introduce soft labels when calculating matching costs in the dynamic label assignment to improve accuracy. Together with better training techniques, the resulting object detector, named RTMDet, achieves 52.8% AP on COCO with 300+ FPS on an NVIDIA 3090 GPU, outperforming the current mainstream industrial detectors. RTMDet achieves the best parameter-accuracy trade-off with tiny/small/medium/large/extra-large model sizes for various application scenarios, and obtains new state-of-the-art performance on real-time instance segmentation and rotated object detection. We hope the experimental results can provide new insights into designing versatile real-time object detectors for many object recognition tasks. Code and models are released at https://github.com/open-mmlab/mmdetection/tree/3.x/configs/rtmdet.

Authors

Kai Chen

11 papers

Wenwei Zhang

5 papers

Chengqi Lyu

1 papers

TL;DR

Abstract

Authors

References101 items

YOLOv6: A Single-Stage Object Detection Framework for Industrial Applications

Advancing Plain Vision Transformer Toward Remote Sensing Foundation Model

YOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors

Shape-Adaptive Selection and Measurement for Oriented Object Detection

Generalized Focal Loss: Towards Efficient Representation Learning for Dense Object Detection

Re-parameterizing Your Optimizers rather than Architectures

MMRotate: A Rotated Object Detection Benchmark using PyTorch

PP-YOLOE: An evolved version of YOLO

Sparse Instance Activation for Real-Time Instance Segmentation

Scaling Up Your Kernels to 31×31: Revisiting Large Kernel Design in CNNs

GiraffeDet: A Heavy-Neck Paradigm for Object Detection

The KFIoU Loss for Rotated Object Detection

A ConvNet for the 2020s

Masked Autoencoders Are Scalable Vision Learners

ResNet strikes back: An improved training procedure in timm

TOOD: Task-aligned One-stage Object Detection

Oriented R-CNN for Object Detection

YOLOX: Exceeding YOLO Series in 2021

K-Net: Towards Unified Image Segmentation

Learning High-Precision Bounding Box for Rotated Object Detection via Kullback-Leibler Divergence

Beyond Bounding-Box: Convex-hull Feature Adaptation for Oriented and Densely Packed Object Detection

Oriented RepPoints for Aerial Object Detection

OTA: Optimal Transport Assignment for Object Detection

ReDet: A Rotation-equivariant Detector for Aerial Object Detection

Rethinking Rotated Object Detection with Gaussian Wasserstein Distance Loss

RepVGG: Making VGG-style ConvNets Great Again

Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation

Dense Label Encoding for Boundary Discontinuity Free Rotation Detection

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Align Deep Features for Oriented Object Detection

Probabilistic Anchor Assignment with IoU Prediction for Object Detection

End-to-End Object Detection with Transformers

YOLOv4: Optimal Speed and Accuracy of Object Detection

Feature Pyramid Grids

SOLOv2: Dynamic, Faster and Stronger

Conditional Convolutions for Instance Segmentation

Arbitrary-Oriented Object Detection with Circular Smooth Label

SpineNet: Learning Scale-Permuted Backbone for Recognition and Localization

SOLO: Segmenting Objects by Locations

Bridging the Gap Between Anchor-Based and Anchor-Free Detection via Adaptive Training Sample Selection

CSPNet: A New Backbone that can Enhance Learning Capability of CNN

Gliding Vertex on the Horizontal Bounding Box for Multi-Oriented Object Detection

EfficientDet: Scalable and Efficient Object Detection

R3Det: Refined Single-Stage Detector with Feature Refinement for Rotating Object

IoU Loss for 2D/3D Object Detection

NAS-FCOS: Fast Neural Architecture Search for Object Detection

Instance Segmentation by Jointly Optimizing Spatial Embeddings and Clustering Bandwidth

Learning RoI Transformer for Oriented Object Detection in Aerial Images

CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features

An Energy and GPU-Computation Efficient Backbone Network for Real-Time Object Detection

Objects as Points

NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection

YOLACT: Real-Time Instance Segmentation

Libra R-CNN: Towards Balanced Learning for Object Detection

FCOS: Fully Convolutional One-Stage Object Detection

DetNAS: Backbone Search for Object Detection

Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression

MultiGrain: a unified image embedding for classes and instances

Bag of Freebies for Training Object Detection Neural Networks

Hybrid Task Cascade for Instance Segmentation

Bag of Tricks for Image Classification with Convolutional Neural Networks

YOLOv3: An Incremental Improvement

Group Normalization

Path Aggregation Network for Instance Segmentation

Cascade R-CNN: Delving Into High Quality Object Detection

DOTA: A Large-Scale Dataset for Object Detection in Aerial Images

Non-local Neural Networks

Decoupled Weight Decay Regularization

mixup: Beyond Empirical Risk Minimization

Focal Loss for Dense Object Detection

Dilated Residual Networks

MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications

Mask R-CNN

YOLO9000: Better, Faster, Stronger

Feature Pyramid Networks for Object Detection

Understanding the Effective Receptive Field in Deep Convolutional Neural Networks