Multi-view 3D Object Detection Network for Autonomous Driving

Published in

Computer Vision and Pattern Recognition(2016)

External Links:

Generate Graph DownloadPDF

TL;DR

This paper proposes Multi-View 3D networks (MV3D), a sensory-fusion framework that takes both LIDAR point cloud and RGB images as input and predicts oriented 3D bounding boxes and designs a deep fusion scheme to combine region-wise features from multiple views and enable interactions between intermediate layers of different paths.

Abstract

This paper aims at high-accuracy 3D object detection in autonomous driving scenario. We propose Multi-View 3D networks (MV3D), a sensory-fusion framework that takes both LIDAR point cloud and RGB images as input and predicts oriented 3D bounding boxes. We encode the sparse 3D point cloud with a compact multi-view representation. The network is composed of two subnetworks: one for 3D object proposal generation and another for multi-view feature fusion. The proposal network generates 3D candidate boxes efficiently from the birds eye view representation of 3D point cloud. We design a deep fusion scheme to combine region-wise features from multiple views and enable interactions between intermediate layers of different paths. Experiments on the challenging KITTI benchmark show that our approach outperforms the state-of-the-art by around 25% and 30% AP on the tasks of 3D localization and 3D detection. In addition, for 2D detection, our approach obtains 14.9% higher AP than the state-of-the-art on the hard data among the LIDAR-based methods.

Authors

Ji Wan

2 papers

Xiaozhi Chen

1 papers

Huimin Ma

1 papers

References34 items

On-Board Object Detection: Multicue, Multimodal, and Multiview Random Forest of Local Experts

3D fully convolutional network for vehicle detection in point cloud

Vote3Deep: Fast object detection in 3D point clouds using efficient convolutional neural networks

3D Object Proposals Using Stereo Imagery for Accurate Object Class Detection

A Unified Multi-scale Deep Convolutional Neural Network for Fast Object Detection

Multi-view 3D Object Detection Network for Autonomous Driving

Published in

Computer Vision and Pattern Recognition(2016)

External Links:

Generate Graph DownloadPDF

TL;DR

Abstract

Authors

Ji Wan

2 papers

Xiaozhi Chen

1 papers

Huimin Ma

1 papers

References34 items

On-Board Object Detection: Multicue, Multimodal, and Multiview Random Forest of Local Experts

3D fully convolutional network for vehicle detection in point cloud

Vote3Deep: Fast object detection in 3D point clouds using efficient convolutional neural networks

3D Object Proposals Using Stereo Imagery for Accurate Object Class Detection

A Unified Multi-scale Deep Convolutional Neural Network for Fast Object Detection

FusionNet: 3D Object Classification Using Multiple Data Representations

Monocular 3D Object Detection for Autonomous Driving

Learning with Side Information through Modality Hallucination

Vehicle Detection from 3D Lidar Using Fully Convolutional Network

Exploit All the Layers: Fast and Accurate CNN Object Detector with Scale Dependent Pooling and Cascaded Rejection Classifiers

Deeply-Fused Nets

FractalNet: Ultra-Deep Neural Networks without Residuals

A Continuous Occlusion Model for Road Scene Understanding

Subcategory-Aware Convolutional Neural Networks for Object Proposals and Detection

Volumetric and Multi-view CNNs for Object Classification on 3D Data

3D Object Proposals for Accurate Object Class Detection

Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Images

Voting for Voting in Online Point Cloud Object Detection

Joint SFM and detection cues for monocular 3D localization in road scenes

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

Data-driven 3D Voxel Patterns for object category recognition

Multi-view Convolutional Neural Networks for 3D Shape Recognition

What Makes for Effective Detection Proposals?

Edge Boxes: Locating Object Proposals from Edges

Sliding Shapes for 3D Object Detection in Depth Images

Very Deep Convolutional Networks for Large-Scale Image Recognition

Are Cars Just 3D Boxes? Jointly Estimating the 3D Shape of Multiple Objects

Detailed 3D Representations for Object Recognition and Modeling

CPMC: Automatic Object Segmentation Using Constrained Parametric Min-Cuts

Are we ready for autonomous driving? The KITTI vision benchmark suite

Segmentation as selective search for object recognition

A Multilevel Mixture-of-Experts Framework for Pedestrian Classification

Field of Study

Computer Science

Journal Information

Name

2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Page

1800-1807

Venue Information

Name

Computer Vision and Pattern Recognition

Type

conference

URL

https://ieeexplore.ieee.org/xpl/conhome.jsp?punumber=1000147

Alternate Names

CVPR
Comput Vis Pattern Recognit

TL;DR

Abstract

Authors

References34 items

On-Board Object Detection: Multicue, Multimodal, and Multiview Random Forest of Local Experts

3D fully convolutional network for vehicle detection in point cloud

Vote3Deep: Fast object detection in 3D point clouds using efficient convolutional neural networks

3D Object Proposals Using Stereo Imagery for Accurate Object Class Detection

A Unified Multi-scale Deep Convolutional Neural Network for Fast Object Detection

TL;DR

Abstract

Authors

References34 items

On-Board Object Detection: Multicue, Multimodal, and Multiview Random Forest of Local Experts

3D fully convolutional network for vehicle detection in point cloud

Vote3Deep: Fast object detection in 3D point clouds using efficient convolutional neural networks

3D Object Proposals Using Stereo Imagery for Accurate Object Class Detection

A Unified Multi-scale Deep Convolutional Neural Network for Fast Object Detection

FusionNet: 3D Object Classification Using Multiple Data Representations

Monocular 3D Object Detection for Autonomous Driving

Learning with Side Information through Modality Hallucination

Vehicle Detection from 3D Lidar Using Fully Convolutional Network

Exploit All the Layers: Fast and Accurate CNN Object Detector with Scale Dependent Pooling and Cascaded Rejection Classifiers

Deeply-Fused Nets

FractalNet: Ultra-Deep Neural Networks without Residuals

A Continuous Occlusion Model for Road Scene Understanding

Subcategory-Aware Convolutional Neural Networks for Object Proposals and Detection

Volumetric and Multi-view CNNs for Object Classification on 3D Data

3D Object Proposals for Accurate Object Class Detection

Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Images

Voting for Voting in Online Point Cloud Object Detection

Joint SFM and detection cues for monocular 3D localization in road scenes

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

Data-driven 3D Voxel Patterns for object category recognition

Multi-view Convolutional Neural Networks for 3D Shape Recognition

Fast R-CNN

What Makes for Effective Detection Proposals?

Edge Boxes: Locating Object Proposals from Edges

Sliding Shapes for 3D Object Detection in Depth Images

Very Deep Convolutional Networks for Large-Scale Image Recognition

Are Cars Just 3D Boxes? Jointly Estimating the 3D Shape of Multiple Objects

Detailed 3D Representations for Object Recognition and Modeling

CPMC: Automatic Object Segmentation Using Constrained Parametric Min-Cuts

Are we ready for autonomous driving? The KITTI vision benchmark suite

Segmentation as selective search for object recognition

A Multilevel Mixture-of-Experts Framework for Pedestrian Classification

Γ and B

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names