End-to-End Object Detection with Transformers (2020-05-26T00:00:00.000000Z)

TL;DR

This work presents a new method that views object detection as a direct set prediction problem, and demonstrates accuracy and run-time performance on par with the well-established and highly-optimized Faster RCNN baseline on the challenging COCO object detection dataset.

Abstract

We present a new method that views object detection as a direct set prediction problem. Our approach streamlines the detection pipeline, effectively removing the need for many hand-designed components like a non-maximum suppression procedure or anchor generation that explicitly encode our prior knowledge about the task. The main ingredients of the new framework, called DEtection TRansformer or DETR, are a set-based global loss that forces unique predictions via bipartite matching, and a transformer encoder-decoder architecture. Given a fixed small set of learned object queries, DETR reasons about the relations of the objects and the global image context to directly output the final set of predictions in parallel. The new model is conceptually simple and does not require a specialized library, unlike many other modern detectors. DETR demonstrates accuracy and run-time performance on par with the well-established and highly-optimized Faster RCNN baseline on the challenging COCO object detection dataset. Moreover, DETR can be easily generalized to produce panoptic segmentation in a unified manner. We show that it significantly outperforms competitive baselines. Training code and pretrained models are available at this https URL.

Authors

Gabriel Synnaeve

13 papers

Francisco Massa

4 papers

Sergey Zagoruyko

3 papers

TL;DR

Abstract

Authors

References53 items

Imputer: Sequence Modelling via Imputation and Dynamic Programming

Bridging the Gap Between Anchor-Based and Anchor-Free Detection via Adaptive Training Sample Selection

PyTorch: An Imperative Style, High-Performance Deep Learning Library

End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures

On the Relationship between Self-Attention and Convolutional Layers

Cascade R-CNN: High Quality Object Detection and Instance Segmentation

RWTH ASR Systems for LibriSpeech: Hybrid vs Attention - w/o Data Augmentation

Attention Augmented Convolutional Networks

Objects as Points

Elucidating image-to-set prediction: An analysis of models, losses and datasets

FCOS: Fully Convolutional One-Stage Object Detection

Mask-Predict: Parallel Decoding of Conditional Masked Language Models

Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression

UPSNet: A Unified Panoptic Segmentation Network

Panoptic Feature Pyramid Networks

Rethinking ImageNet Pre-Training

Character-Level Language Modeling with Deeper Self-Attention

Deep Perm-Set Net: Learn to predict sets with unknown permutation and cardinality using deep neural networks

Image Transformer

Panoptic Segmentation

Recurrent Neural Networks for Semantic Instance Segmentation

Relation Networks for Object Detection

Parallel WaveNet: Fast High-Fidelity Speech Synthesis

Non-local Neural Networks

Decoupled Weight Decay Regularization

Non-Autoregressive Neural Machine Translation

Focal Loss for Dense Object Detection

Attention is All you Need

Learning Non-maximum Suppression

Soft-NMS — Improving Object Detection with One Line of Code

Feature Pyramid Networks for Object Detection

DeepSetNet: Predicting Sets with Deep Neural Networks

Fully Convolutional Instance-Aware Semantic Segmentation

V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation

End-to-End Instance Segmentation with Recurrent Attention

Deep Residual Learning for Image Recognition

SSD: Single Shot MultiBox Detector

Recurrent Instance Segmentation

Order Matters: Sequence to sequence for sets

Learning to decompose for object detection and instance segmentation

End-to-End People Detection in Crowded Scenes

You Only Look Once: Unified, Real-Time Object Detection

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

Sequence to Sequence Learning with Neural Networks

Neural Machine Translation by Jointly Learning to Align and Translate

Microsoft COCO: Common Objects in Context

Scalable Object Detection Using Deep Neural Networks

Understanding the difficulty of training deep feedforward neural networks

The Hungarian method for the assignment problem

Detectron2

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Language Models are Unsupervised Multitask Learners

Mask R-CNN

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names