Center-based 3D Object Detection and Tracking (2020-06-19T00:00:00.000000Z)

TL;DR

The framework, CenterPoint, first detects centers of objects using a keypoint detector and regresses to other attributes, including 3D size, 3D orientation, and velocity, and refines these estimates using additional point features on the object.

Abstract

Three-dimensional objects are commonly represented as 3D boxes in a point-cloud. This representation mimics the well-studied image-based 2D bounding-box detection but comes with additional challenges. Objects in a 3D world do not follow any particular orientation, and box-based detectors have difficulties enumerating all orientations or fitting an axis-aligned bounding box to rotated objects. In this paper, we instead propose to represent, detect, and track 3D objects as points. Our framework, CenterPoint, first detects centers of objects using a keypoint detector and regresses to other attributes, including 3D size, 3D orientation, and velocity. In a second stage, it refines these estimates using additional point features on the object. In CenterPoint, 3D object tracking simplifies to greedy closest-point matching. The resulting detection and tracking algorithm is simple, efficient, and effective. CenterPoint achieved state-of-the-art performance on the nuScenes benchmark for both 3D detection and tracking, with 65.5 NDS and 63.8 AMOTA for a single model. On the Waymo Open Dataset, Center-Point outperforms all previous single model methods by a large margin and ranks first among all Lidar-only submissions. The code and pretrained models are available at https://github.com/tianweiy/CenterPoint.

Authors

Philipp Krähenbühl

10 papers

Xingyi Zhou

4 papers

Tianwei Yin

2 papers

TL;DR

Abstract

Authors

References70 items

An LSTM Approach to Temporal 3D Object Detection in LiDAR Point Clouds

Pillar-based Object Detection for Autonomous Driving

1st Place Solution for Waymo Open Dataset Challenge - 3D Detection and Domain Adaptation

AFDet: Anchor Free One Stage 3D Object Detection

Structure Aware Single-Stage 3D Object Detection From Point Cloud

Range Conditioned Dilated Convolutions for Scale Invariant 3D Object Detection

Learning to Evaluate Perception Models Using Planner-Centric Metrics

SSN: Shape Signature Networks for Multi-class Object Detection from Point Clouds

LiDAR-Based Online 3D Video Object Detection With Graph-Based Message Passing and Spatiotemporal Transformer Attention

Tracking Objects as Points

DOPS: Learning to Detect 3D Objects and Predict Their 3D Shapes

3DSSD: Point-Based 3D Single Stage Object Detector

Probabilistic 3D Multi-Object Tracking for Autonomous Driving

PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection

Object as Hotspots: An Anchor-Free 3D Object Detection Approach via Firing of Hotspots

What You See is What You Get: Exploiting Visibility for 3D Object Detection

Scalability in Perception for Autonomous Driving: An Open Dataset Benchmark

PointPainting: Sequential Fusion for 3D Object Detection

End-to-End Multi-View Fusion for 3D Object Detection in LiDAR Point Clouds

Identifying Unknown Instances for Autonomous Driving

StarNet: Targeted Computation for Object Detection in Point Clouds

Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection

R3Det: Refined Single-Stage Detector with Feature Refinement for Rotating Object

Fast Point R-CNN

Multiple Object Tracking With Attention to Appearance, Structure, Motion and Size

STD: Sparse-to-Dense 3D Object Detector for Point Cloud

3D Multi-Object Tracking: A Baseline and New Evaluation Metrics

A Baseline for 3D Multi-Object Tracking

From Points to Parts: 3D Object Detection From Point Cloud With Part-Aware and Part-Aggregation Network

Multi-Task Multi-Sensor Fusion for 3D Object Detection

Monocular Plan View Networks for Autonomous Driving

Deep Hough Voting for 3D Object Detection in Point Clouds

Objects as Points

nuScenes: A Multimodal Dataset for Autonomous Driving

GS3D: An Efficient 3D Object Detection Framework for Autonomous Driving

Tracking Without Bells and Whistles

PointPillars: Fast Encoders for Object Detection From Point Clouds

PointRCNN: 3D Object Proposal Generation and Detection From Point Cloud

ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst

SCRDet: Towards More Robust Detection for Small, Cluttered and Rotated Objects

SECOND: Sparsely Embedded Convolutional Detection

CornerNet: Detecting Objects as Paired Keypoints

Acquisition of Localization Confidence for Accurate Object Detection

PIXOR: Real-time 3D Object Detection from Point Clouds

Complex-YOLO: An Euler-Region-Proposal for Real-Time 3D Object Detection on Point Clouds

3D Semantic Segmentation with Submanifold Sparse Convolutional Networks

Frustum PointNets for 3D Object Detection from RGB-D Data

VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection

Decoupled Weight Decay Regularization

Focal Loss for Dense Object Detection

PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space

Simple online and realtime tracking with a deep association metric

Mask R-CNN

Deformable Convolutional Networks

YOLO9000: Better, Faster, Stronger

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation

Vote3Deep: Fast object detection in 3D point clouds using efficient convolutional neural networks

Simple online and realtime tracking

SSD: Single Shot MultiBox Detector

Voting for Voting in Online Point Cloud Object Detection

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

Fast R-CNN

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation

Improving neural networks by preventing co-adaptation of feature detectors

Are we ready for autonomous driving? The KITTI vision benchmark suite

Every View Counts: Cross-View Consistency in 3D Object Detection with Hybrid-Cylindrical-Spherical Voxelization

The 1cycle policy

Multiple Object Tracking Performance Metrics and Evaluation in a Smart Room Environment

Edinburgh Research Explorer The PASCAL Visual Object Classes (VOC) Challenge

Field of Study

Journal Information

Name

Page

Venue Information

Name