DETRs with Hybrid Matching (2022-07-26T00:00:00.000000Z)

TL;DR

A simple yet effective method based on a hybrid matching scheme that combines the original one-to-one matching branch with an auxiliary one- to-many matching branch during training that shows that a wide range of representative DETR methods can be consistently improved across a widerange of visual tasks.

Abstract

One-to-one set matching is a key design for DETR to establish its end-to-end capability, so that object detection does not require a hand-crafted NMS (non-maximum suppression) to remove duplicate detections. This end-to-end signature is important for the versatility of DETR, and it has been generalized to broader vision tasks. However, we note that there are few queries assigned as positive samples and the one-to-one set matching significantly reduces the training efficacy of positive samples. We propose a simple yet effective method based on a hybrid matching scheme that combines the original one-to-one matching branch with an auxiliary one-to-many matching branch during training. Our hybrid strategy has been shown to significantly improve accuracy. In inference, only the original one-to-one match branch is used, thus maintaining the end-to-end merit and the same inference efficiency of DETR. The method is namedℋ-DETR, and it shows that a wide range of representative DETR methods can be consistently improved across a wide range of visual tasks, including Deformable-DETR, PETRv2, PETR, and TransTrack, among others. Code is available at: https://github.com/HDETR.

Authors

Hao He

2 papers

Yuhui Yuan

3 papers

Ding Jia

1 papers

TL;DR

Abstract

Authors

References94 items

MOAT: Alternating Mobile Convolution and Attention Brings Strong Vision Models

CF-DETR: Coarse-to-Fine Transformers for End-to-End Object Detection

Anchor DETR: Query Design for Transformer-Based Detector

Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentation

PETRv2: A Unified Framework for 3D Perception from Multi-Camera Images

End-to-End Multi-Person Pose Estimation with Transformers

Lite-MDETR: A Lightweight Multi-Modal Detector

CMT-DeepLab: Clustering Mask Transformers for Panoptic Segmentation

BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework

FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness

BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation

Text Spotting Transformers

BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimation

BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers

BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detection

Exploring Plain Vision Transformer Backbones for Object Detection

AdaMixer: A Fast-Converging Query-Based Object Detector

Towards End-to-End Unified Scene Text Detection and Layout Analysis

DepthFormer: Exploiting Long-range Correlation and Local Information for Accurate Monocular Depth Estimation

TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers

Towards Data-Efficient Detection Transformers

Accelerating DETR Convergence via Semantic-Aligned Matching

PETR: Position Embedding Transformation for Multi-View 3D Object Detection

DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection

DN-DETR: Accelerate DETR Training by Introducing Query DeNoising

Arbitrary Shape Text Detection using Transformers

DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR

A ConvNet for the 2020s

Detecting Twenty-thousand Classes using Image-level Supervision

Language as Queries for Referring Video Object Segmentation

BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View

SOIT: Segmenting Objects with Instance-Aware Transformers

Mask2Former for Video Instance Segmentation

LAVT: Language-Aware Vision Transformer for Referring Image Segmentation

Masked-attention Mask Transformer for Universal Image Segmentation

MViTv2: Improved Multiscale Vision Transformers for Classification and Detection

Sparse DETR: Efficient End-to-End Object Detection with Learnable Sparsity

Swin Transformer V2: Scaling Up Capacity and Resolution

Direct Multi-view Multi-person 3D Pose Estimation

DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries

The Center of Attention: Center-Keypoint Grouping via Attention for Multi-Person Pose Estimation

Dynamic DETR: End-to-End Object Detection with Dynamic Attention

An End-to-End Transformer Model for 3D Object Detection

Panoptic SegFormer: Delving Deeper into Panoptic Segmentation with Transformers

Conditional DETR for Fast Training Convergence

CBNet: A Composite Backbone Network Architecture for Object Detection

SOLQ: Segmenting Objects by Learning Queries

Transformer-based Text Detection in the Wild

TrTr: Visual Tracking with Transformer

MOTR: End-to-End Multiple-Object Tracking with TRansformer

Instances as Queries

MDETR - Modulated Detection for End-to-End Multi-Modal Understanding

Pose Recognition with Cascade Transformers

TokenPose: Learning Keypoint Tokens for Human Pose Estimation

Efficient DETR: Improving End-to-End Object Detector with Dense Prior

Group-Free 3D Object Detection via Transformers

Learning Spatio-Temporal Transformer for Visual Tracking

Transformer Tracking

TFPose: Direct Human Pose Estimation with Transformers

End-to-End Trainable Multi-Instance Pose Estimation with Transformers

Depth from Camera Motion and Object Detection

Fast Convergence of DETR with Spatially Modulated Co-Attention

TrackFormer: Multi-Object Tracking with Transformers

Line Segment Detection Using Transformers without Edges

TransTrack: Multiple-Object Tracking with Transformer

Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation

End-to-End Object Detection with Fully Convolutional Network

MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers

One Metric to Measure Them All: Localisation Recall Precision (LRP) for Evaluating Visual Detection Tasks

Deformable DETR: Deformable Transformers for End-to-End Object Detection

AutoAssign: Differentiable Label Assignment for Dense Object Detection

End-to-End Object Detection with Transformers

Bridging the Gap Between Anchor-Based and Anchor-Free Detection via Adaptive Training Sample Selection

CenterMask: Real-Time Anchor-Free Instance Segmentation

Object-Contextual Representations for Semantic Segmentation

LVIS: A Dataset for Large Vocabulary Instance Segmentation