FairMOT: On the Fairness of Detection and Re-identification in Multiple Object Tracking (2020-04-04T00:00:00.000000Z)

TL;DR

This work presents a simple yet effective approach termed as FairMOT based on the anchor-free object detection architecture CenterNet, which outperforms the state-of-the-art methods by a large margin on several public datasets.

Abstract

Multi-object tracking (MOT) is an important problem in computer vision which has a wide range of applications. Formulating MOT as multi-task learning of object detection and re-ID in a single network is appealing since it allows joint optimization of the two tasks and enjoys high computation efficiency. However, we find that the two tasks tend to compete with each other which need to be carefully addressed. In particular, previous works usually treat re-ID as a secondary task whose accuracy is heavily affected by the primary detection task. As a result, the network is biased to the primary detection task which is not fair to the re-ID task. To solve the problem, we present a simple yet effective approach termed as FairMOT based on the anchor-free object detection architecture CenterNet. Note that it is not a naive combination of CenterNet and re-ID. Instead, we present a bunch of detailed designs which are critical to achieve good tracking results by thorough empirical studies. The resulting approach achieves high accuracy for both detection and tracking. The approach outperforms the state-of-the-art methods by a large margin on several public datasets. The source code and pre-trained models are released at https://github.com/ifzhang/FairMOT.

Authors

Xinggang Wang

22 papers

Wenyu Liu

13 papers

Yifu Zhang

2 papers

TL;DR

Abstract

Authors

References115 items

ByteTrack: Multi-Object Tracking by Associating Every Detection Box

VoxelTrack: Multi-Person 3D Human Pose Estimation and Tracking in the Wild

Local Metrics for Multi-Object Tracking

TransTrack: Multiple-Object Tracking with Transformer

What Makes for End-to-End Object Detection?

Sparse R-CNN: End-to-End Object Detection with Learnable Proposals

Rethinking the Competition Between Detection and ReID in Multiobject Tracking

FGAGT: Flow-Guided Adaptive Graph Tracking

MAT: Motion-Aware Multi-Object Tracking

Chained-Tracker: Chaining Paired Attentive Regression Results for End-to-End Joint Multiple-Object Detection and Tracking

Lifted Disjoint Paths with Application in Multiple Object Tracking

Quasi-Dense Similarity Learning for Multiple Object Tracking

TubeTK: Adopting Tubes to Track Multi-Object in a One-Step Training Model

Long-Term Tracking With Deep Tracklet Association

Multi-Task Learning for Dense Prediction Tasks: A Survey

Distilling Localization for Self-Supervised Representation Learning

Tracking Objects as Points

Designing Network Design Spaces

RetinaTrack: Online Single Stage Joint Detection and Tracking

CentripetalNet: Pursuing High-Quality Keypoint Pairs for Object Detection

MOT20: A benchmark for multi object tracking in crowded scenes

Learning a Neural Solver for Multiple Object Tracking

Model Adaption Object Detection System for Robot

Towards Real-Time Multi-Object Tracking

HarDNet: A Low Memory Traffic Network

HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation

Deep High-Resolution Representation Learning for Visual Recognition

Multiple People Tracking Using Body and Joint Detections

Spatial-Temporal Relation Networks for Multi-Object Tracking

RepPoints: Point Set Representation for Object Detection

CenterNet: Keypoint Triplets for Object Detection

Objects as Points

FAMNet: Joint Learning of Feature, Affinity and Multi-Dimensional Assignment for Online Multiple Object Tracking

Bag of Tricks and a Strong Baseline for Deep Person Re-Identification

Tracking Without Bells and Whistles

MOTS: Multi-Object Tracking and Segmentation

Bottom-Up Object Detection by Grouping Extreme and Center Points

Online Multi-Object Tracking With Instance-Aware Tracker and Dynamic Model Refreshment

Detect or Track: Towards Cost-Effective Video Object Detection/Tracking

Deep Affinity Network for Multiple Object Tracking

Multi-Task Learning as Multi-Objective Optimization

Multi-Object Tracking Using Online Metric Learning with Long Short-Term Memory

Online Multi-Object Tracking with Dual Matching Attention Networks

Dynamic Task Prioritization for Multitask Learning

Multi-target tracking using CNN-based features: CNNMTT

CornerNet: Detecting Objects as Paired Keypoints

Online Multi-Target Tracking with Tensor-Based High-Order Graph Matching

Real-Time Multiple People Tracking with Deeply Learned Candidate Selection and Person Re-Identification

CrowdHuman: A Benchmark for Detecting Human in a Crowd

YOLOv3: An Incremental Improvement

End-To-End Multi-Task Learning With Attention

Confidence-Based Data Association and Discriminative Deep Appearance Learning for Robust Online Multi-Object Tracking

Object Detection in Videos by High Quality Object Linking

Cascade R-CNN: Delving Into High Quality Object Detection

Recurrent Autoregressive Networks for Online Multi-object Tracking

GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks

Detect to Track and Track to Detect

Online multi-object tracking with convolutional neural networks

Focal Loss for Dense Object Detection

High-Speed tracking-by-detection without using image information

Deep Layer Aggregation

Multiple People Tracking by Lifted Multicut and Person Re-identification

Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics

In Defense of the Triplet Loss for Person Re-Identification

Simple online and realtime tracking with a deep association metric

Object Detection in Videos with Tubelet Proposal Networks

CityPersons: A Diverse Dataset for Pedestrian Detection

Learning Discriminative Activated Simplices for Action Recognition

Tracking the Untrackable: Learning to Track Multiple Cues with Long-Term Dependencies

Feature Pyramid Networks for Object Detection

A Discriminatively Learned CNN Embedding for Person Reidentification

POI: Multiple Object Tracking with High Performance Detection and Appearance Feature

Online Multi-target Tracking with Strong and Weak Detections

UberNet: Training a Universal Convolutional Neural Network for Low-, Mid-, and High-Level Vision Using Diverse Datasets and Limited Memory

Performance Measures and a Data Set for Multi-target, Multi-camera Tracking

Exploit All the Layers: Fast and Accurate CNN Object Detector with Scale Dependent Pooling and Cascaded Rejection Classifiers