YOLOP: You Only Look Once for Panoptic Driving Perception (2021-08-25T00:00:00.000000Z)

TL;DR

To the best knowledge, this is the first work that can process these three visual perception tasks simultaneously in real-time on an embedded device Jetson TX2(23 FPS), and maintain excellent accuracy.

Abstract

A panoptic driving perception system is an essential part of autonomous driving. A high-precision and real-time perception system can assist the vehicle in making reasonable decisions while driving. We present a panoptic driving perception network (you only look once for panoptic (YOLOP)) to perform traffic object detection, drivable area segmentation, and lane detection simultaneously. It is composed of one encoder for feature extraction and three decoders to handle the specific tasks. Our model performs extremely well on the challenging BDD100K dataset, achieving state-of-the-art on all three tasks in terms of accuracy and speed. Besides, we verify the effectiveness of our multi-task learning model for joint training via ablative studies. To our best knowledge, this is the first work that can process these three visual perception tasks simultaneously in real-time on an embedded device Jetson TX2(23 FPS), and maintain excellent accuracy. To facilitate further research, the source codes and pre-trained models are released at https://github.com/hustvl/YOLOP.

Authors

Xinggang Wang

22 papers

X. Bai

25 papers

Wenqing Cheng

2 papers

TL;DR

Abstract

Authors

References39 items

Location-Sensitive Visual Recognition with Cross-IOU Loss

Using Channel-Wise Attention for Deep CNN Based Real-Time Semantic Segmentation With Class-Aware Edge Information

Scaled-YOLOv4: Scaling Cross Stage Partial Network

DLT-Net: Joint Detection of Drivable Areas, Lane Lines, and Traffic Objects

Ultra Fast Structure-aware Deep Lane Detection

YOLOv4: Optimal Speed and Accuracy of Object Detection

CSPNet: A New Backbone that can Enhance Learning Capability of CNN

Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression

Learning Lightweight Lane Detection CNNs by Self Attention Distillation

Geometric Constrained Joint Lane Segmentation and Lane Boundary Detection

BDD100K: A Diverse Driving Video Database with Scalable Annotation Tooling

YOLOv3: An Incremental Improvement

Path Aggregation Network for Instance Segmentation

Towards End-to-End Lane Detection: an Instance Segmentation Approach

Spatial As Deep: Spatial CNN for Traffic Scene Understanding

Focal Loss for Dense Object Detection

Mask R-CNN

YOLO9000: Better, Faster, Stronger

MultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving

Feature Pyramid Networks for Object Detection

Pyramid Scene Parsing Network

SGDR: Stochastic Gradient Descent with Warm Restarts

ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation

DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

R-FCN: Object Detection via Region-based Fully Convolutional Networks

Deep Residual Learning for Image Recognition

SSD: Single Shot MultiBox Detector

You Only Look Once: Unified, Real-Time Object Detection

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

U-Net: Convolutional Networks for Biomedical Image Segmentation

Fast R-CNN

Fully convolutional networks for semantic segmentation

Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation

Learning with Whom to Share in Multi-task Feature Learning

MultiNet: connecting to multiple IEEE 802.11 networks using a single wireless card

He is now a professor of School of Electronic Information and Communications, HUST, China. His research interests include computer vision, multimedia, and machine learning

She is currently a professor and associate dean with School of Electronic Information and Communications, HUST, China

Θ ← Θ ∪ { θ seg } \ { θ det , θ enc }

Field of Study

Journal Information

Name

Page

Volume

Venue Information

Name

Type

URL

Alternate Names