YOLO3D: End-to-end real-time 3D Oriented Object Bounding Box Detection from LiDAR Point Cloud (2018-08-07T00:00:00.000000Z)

TL;DR

The main contribution is in extending the loss function of YOLO v2 to include the yaw angle, the 3D box center in Cartesian coordinates and the height of the box as a direct regression problem, which enables real-time performance, which is essential for automated driving.

Abstract

Object detection and classification in 3D is a key task in Automated Driving (AD). LiDAR sensors are employed to provide the 3D point cloud reconstruction of the surrounding environment, while the task of 3D object bounding box detection in real time remains a strong algorithmic challenge. In this paper, we build on the success of the one-shot regression meta-architecture in the 2D perspective image space and extend it to generate oriented 3D object bounding boxes from LiDAR point cloud. Our main contribution is in extending the loss function of YOLO v2 to include the yaw angle, the 3D box center in Cartesian coordinates and the height of the box as a direct regression problem. This formulation enables real-time performance, which is essential for automated driving. Our results are showing promising figures on KITTI benchmark, achieving real-time performance (40 fps) on Titan X GPU.

Authors

Ahmad El Sallab

3 papers

Waleed Ali

1 papers

Sherif Abdelkarim

1 papers

YOLO3D: End-to-end real-time 3D Oriented Object Bounding Box Detection from LiDAR Point Cloud

TL;DR

Abstract

Authors

References24 items

YOLOv3: An Incremental Improvement

Complex-YOLO: Real-time 3D Object Detection on Point Clouds

Tiny SSD: A Tiny Single-Shot Detection Deep Convolutional Neural Network for Real-Time Embedded Object Detection

Joint 3D Proposal Generation and Object Detection from View Aggregation

Frustum PointNets for 3D Object Detection from RGB-D Data

VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection

DepthCN: Vehicle detection using 3D-LIDAR and ConvNet

ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices

MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications

Deep MANTA: A Coarse-to-Fine Many-Task Network for Joint 2D and 3D Vehicle Analysis from Monocular Image

YOLO9000: Better, Faster, Stronger

3D Bounding Box Estimation Using Deep Learning and Geometry

3D fully convolutional network for vehicle detection in point cloud

Multi-view 3D Object Detection Network for Autonomous Driving

Vote3Deep: Fast object detection in 3D point clouds using efficient convolutional neural networks

3D Object Proposals Using Stereo Imagery for Accurate Object Class Detection

Vehicle Detection from 3D Lidar Using Fully Convolutional Network

SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size

SSD: Single Shot MultiBox Detector

You Only Look Once: Unified, Real-Time Object Detection

We added terms in the loss function for yaw, z center coordinate, and height regressions to facilitate the 3D oriented bounding box detection

We modiﬁed one max-pooling layer to change the down-sampling from 32 to 16 so we can have a larger grid at the end; this has a contribution in

We removed the skip connection from the model as we found it resulting in less accurate results

Our input consists of 2 channels, one representing the maximum height, and the other one representing the density of points in the point cloud, computed as shown in Eq. (1)

Field of Study