Multi-modal visual tracking: Review and experimental comparison (2020-12-08T00:00:00.000000Z)

TL;DR

Different aspects of multi-modal tracking algorithms are summarized under a unified taxonomy, with specific focus on visible-depth (RGB-D) and visible-thermal (RGB-T) tracking.

Abstract

Visual object tracking has been drawing increasing attention in recent years, as a fundamental task in computer vision. To extend the range of tracking applications, researchers have been introducing information from multiple modalities to handle specific scenes, with promising research prospects for emerging methods and benchmarks. To provide a thorough review of multi-modal tracking, different aspects of multi-modal tracking algorithms are summarized under a unified taxonomy, with specific focus on visible-depth (RGB-D) and visible-thermal (RGB-T) tracking. Subsequently, a detailed description of the related benchmarks and challenges is provided. Extensive experiments were conducted to analyze the effectiveness of trackers on five datasets: PTB, VOT19-RGBD, GTOT, RGBT234, and VOT19-RGBT. Finally, various future directions, including model design and dataset construction, are discussed from different perspectives for further research.

Authors

Huchuan Lu

21 papers

Dong Wang

4 papers

Pengyu Zhang

2 papers

TL;DR

Abstract

Authors

References142 items

RGBD1K: A Large-scale Dataset and Benchmark for RGB-D Object Tracking

Attribute-Based Progressive Fusion Network for RGBT Tracking

Learning reliable modal weight with transformer for robust RGBT tracking

Visible-Thermal UAV Tracking: A Large-Scale Benchmark and New Baseline

DepthTrack: Unveiling the Power of RGBD Tracking

Learning Adaptive Attribute-Driven Representation for Real-Time RGB-T Tracking

LasHeR: A Large-Scale High-Diversity Benchmark for RGBT Tracking

Learning Spatio-Temporal Transformer for Visual Tracking

Transformer Tracking

COMPARISON:

Object fusion tracking based on visible and infrared images: A comprehensive review

The Eighth Visual Object Tracking VOT2020 Challenge Results

Challenge-Aware RGBT Tracking

Jointly Modeling Motion and Appearance Cues for Robust RGB-T Tracking

Cross-Modal Pattern-Propagation for RGB-T Tracking

Robust RGBD Tracking via Weighted Convolution Operators

Quality-Aware Feature Aggregation Network for Robust RGBT Tracking

Modality-correlation-aware sparse representation for RGB-infrared object tracking

Object Tracking in RGB-T Videos Using Modal-Aware Attention Network and Competitive Learning

RGB-D Object Tracking with Occlusion Detection

D3S – A Discriminative Single Shot Segmentation Tracker

Target-Aware Correlation Filter Tracking in RGBD Videos

Deep Adaptive Fusion Network for High Performance RGBT Tracking

The Seventh Visual Object Tracking VOT2019 Challenge Results

Learning Local-Global Multi-Graph Descriptors for RGB-T Object Tracking

Multi-Modal Fusion for End-to-End RGB-T Tracking

Dense Feature Aggregation and Pruning for RGBT Tracking

Multi-Adapter RGBT Tracking

Exploiting depth information to increase object tracking robustness

Multi-Branch Context-Aware Network for Person Re-Identification

CDTB: A Color and Depth Visual Object Tracking Dataset and Benchmark

Thermal infrared and visible sequences fusion tracking based on a hybrid tracking framework with adaptive weighting scheme

Online Non-Negative Multi-Modality Feature Template Learning for RGB-Assisted Infrared Tracking

Depth information aided constrained correlation filter for visual tracking

Context-Aware Three-Dimensional Mean-Shift With Occlusion Handling for Robust Object Tracking in RGB-D Videos

Fast RGB-T Tracking via Cross-Modal Correlation Filters

Learning Modality-Consistency Feature Templates: A Robust RGB-Infrared Tracking System

Deeper and Wider Siamese Networks for Real-Time Visual Tracking

Fast Online Object Tracking and Segmentation: A Unifying Approach

FuCoLoT - A Fully-Correlational Long-Term Tracker

Object Tracking by Reconstruction With View-Specific Discriminative Correlation Filters

FANet: Quality-Aware Feature Aggregation Network for RGB-T Tracking

Two-stage modality-graphs regularized manifold ranking for RGB-T tracking

Learning Multi-Domain Convolutional Network for RGB-T Visual Tracking

How to Make an RGBD Tracker?

Cross-Modal Ranking with Soft Consistency and Noisy Labels for Robust RGB-T Tracking

Robust Fusion of Color and Depth Data for RGB-D Target Tracking Using Adaptive Range-Invariant Depth Models and Spatio-Temporal Consistency Constraints

Learning Collaborative Sparse Correlation Filter for Real-Time Multispectral Object Tracking

DARTS: Differentiable Architecture Search

Synthetic Data Generation for End-to-End Thermal Infrared Tracking

A Real-time RGB-D tracker based on KCF

RGB-T Object Tracking: Benchmark and Baseline

Robust Collaborative Discriminative Learning for RGB-Infrared Tracking

Now you see me: evaluating performance in long-term visual tracking

Learning Spatial-Temporal Regularized Correlation Filters for Visual Tracking

Depth Masked Discriminative Correlation Filter

Infrared and visible image fusion methods and applications: A survey

Fusing two-stream convolutional neural networks for RGB-T object tracking

Deep Multimodal Learning: A Survey on Recent Advances and Trends

Weighted Sparse Representation Regularized Graph Learning for RGB-T Object Tracking

Robust object tracking with RGBD-based sparse learning

Learning Policies for Adaptive Tracking with Deep Feature Cascades

Learning Transferable Architectures for Scalable Image Recognition

Multiple human tracking in RGB-depth data: a survey

Multimodal Machine Learning: A Survey and Taxonomy

Learning Background-Aware Correlation Filters for Visual Tracking

Grayscale-Thermal Object Tracking via Multitask Laplacian Sparse Representation

Learning Collaborative Sparse Representation for Grayscale-Thermal Tracking

Online RGB-D tracking via detection-learning-segmentation

ECO: Efficient Convolution Operators for Tracking

Discriminative Correlation Filter Tracker with Channel and Spatial Reliability

DS-KCF: a real-time tracker for RGB-D data

Discriminative Scale Space Tracking

An occlusion-aware particle filter tracker to handle complex and persistent occlusions

Fully-Convolutional Siamese Networks for Object Tracking

3D Part-Based Sparse Tracker with Automatic Synchronization and Registration