A survey of deep learning techniques for autonomous driving (2019-10-17T00:00:00.000000Z)

TL;DR

The objective of this paper is to survey the current state‐of‐the‐art on deep learning technologies used in autonomous driving, by presenting AI‐based self‐driving architectures, convolutional and recurrent neural networks, as well as the deep reinforcement learning paradigm.

Abstract

The last decade witnessed increasingly rapid progress in self‐driving vehicle technology, mainly backed up by advances in the area of deep learning and artificial intelligence (AI). The objective of this paper is to survey the current state‐of‐the‐art on deep learning technologies used in autonomous driving. We start by presenting AI‐based self‐driving architectures, convolutional and recurrent neural networks, as well as the deep reinforcement learning paradigm. These methodologies form a base for the surveyed driving scene perception, path planning, behavior arbitration, and motion control algorithms. We investigate both the modular perception‐planning‐action pipeline, where each module is built using deep learning methods, as well as End2End systems, which directly map sensory information to steering commands. Additionally, we tackle current challenges encountered in designing AI architectures for autonomous driving, such as their safety, training data sources, and computational hardware. The comparison presented in this survey helps gain insight into the strengths and limitations of deep learning and AI approaches for autonomous driving and assist with design choices.

Authors

S. Grigorescu

1 papers

Bogdan Trasnea

1 papers

Tiberiu T. Cocias

1 papers

TL;DR

Abstract

Authors

References175 items

Object Detection

Comparative Study

NeuroTrajectory: A Neuroevolutionary Approach to Local State Trajectory Learning for Autonomous Vehicles

nuScenes: A Multimodal Dataset for Autonomous Driving

Learning to See the Wood for the Trees: Deep Laser Localization in Urban and Natural Environments on a CPU

Deep Grid Net (DGN): A Deep Learning System for Real-Time Driving Context Understanding

Pseudo-LiDAR From Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving

Deep Reinforcement Learning for Autonomous Driving

RoarNet: A Robust 3D Object Detection based on RegiOn Approximation Refinement

Feature Learning for Scene Flow Estimation from LIDAR

Learning to Localize Using a LiDAR Intensity Map

Leveraging Deep Visual Descriptors for Hierarchical Efficient Localization

CornerNet: Detecting Objects as Paired Keypoints

Intelligent Land-Vehicle Model Transfer Trajectory Planning Method Based on Deep Reinforcement Learning

Learning dexterous in-hand manipulation

Object Detection With Deep Learning: A Review

Learning to Drive in a Day

An Efficient Object Detection Algorithm Based on Compressed Networks

Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net

Planning and Decision-Making for Autonomous Vehicles

End-to-End Race Driving with Deep Reinforcement Learning

BDD100K: A Diverse Driving Video Database with Scalable Annotation Tooling

VLocNet++: Deep Multitask Learning for Semantic Visual Localization and Odometry

YOLOv3: An Incremental Improvement

An Analysis of ISO 26262: Machine Learning and Safety in Automotive Software

End-to-End Learning of Driving Models with Surround-View Cameras and Route Planners

Understanding Performance Differences of FPGAs and GPUs

An Implantable Wireless Neural Interface System for Simultaneous Recording and Stimulation of Peripheral Nerve with a Single Cuff Electrode

DeepPicar: A Low-Cost Deep Neural Network-Based Autonomous Car

Joint 3D Proposal Generation and Object Detection from View Aggregation

Repetitive learning model predictive control: An autonomous racing example

Learning Less is More - 6D Camera Localization via 3D Surface Regression

Frustum PointNets for 3D Object Detection from RGB-D Data

MIT Advanced Vehicle Technology Study: Large-Scale Naturalistic Driving Study of Driver Behavior and Interaction With Automation

Single-Shot Refinement Neural Network for Object Detection

VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection

Driven to Distraction: Self-Supervised Distractor Learning for Robust Monocular Visual Odometry in Urban Environments

Aggressive Deep Driving: Combining Convolutional Neural Networks and Model Predictive Control

End-to-End Deep Learning for Steering Autonomous Vehicles Considering Temporal Dependencies

Rainbow: Combining Improvements in Deep Reinforcement Learning

When to use what data set for your self-driving car algorithm: An overview of publicly available driving datasets

SGN: Sequential Grouping Networks for Instance Segmentation

Agile Off-Road Autonomous Driving Using End-to-End Deep Imitation Learning

Making the Case for Safety of Machine Learning in Highly Automated Driving

An Analysis of ISO 26262: Using Machine Learning Safely in Automotive Software

Simultaneous Localization and Mapping: A Survey of Current Trends in Autonomous Driving

Application of iterative learning control in tracking a Dubin’s path in parallel parking

Camera Relocalization by Computing Pairwise Relative Poses Using Convolutional Neural Network

End-to-End Driving in a Realistic Racing Game with Deep Reinforcement Learning

Aggressive Deep Driving: Model Predictive Control with a CNN Cost Model

A Fast Integrated Planning and Control Framework for Autonomous Driving via Imitation Learning

Dynamic Occupancy Grid Prediction for Urban Autonomous Driving: A Deep Learning Approach with Fully Automatic Labeling

Learning a deep neural net policy for end-to-end control of autonomous vehicles

A novel iterative learning path-tracking control for nonholonomic mobile robots against initial shifts

AdapNet: Adaptive semantic segmentation in adverse environmental conditions

ICNet for Real-Time Semantic Segmentation on High-Resolution Images

Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car

Computer Vision for Autonomous Vehicles: Problems, Datasets and State-of-the-Art

Challenges in Autonomous Vehicle Validation: Keynote Presentation Abstract

Deep Reinforcement Learning framework for Autonomous Driving

Feature analysis and selection for training an end-to-end autonomous vehicle controller using deep learning approach

Image-Based Localization Using Hourglass Networks

Combining neural networks and tree search for task and motion planning in challenging environments

Can FPGAs Beat GPUs in Accelerating Next-Generation Deep Neural Networks?

Perception, Planning, Control, and Coordination for Autonomous Vehicles

Overview of Environment Perception for Intelligent Vehicles

Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks

1 year, 1000 km: The Oxford RobotCar dataset

YOLO9000: Better, Faster, Stronger

Detecting unexpected obstacles for self-driving cars: Fusing deep learning and geometric modeling

End-to-End Learning of Driving Models from Large-Scale Video Datasets

Pyramid Scene Parsing Network

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation

On Human Intellect and Machine Failures: Troubleshooting Integrative Machine Learning Systems

Image-Based Localization Using LSTMs for Structured Feature Correlation

Multi-view 3D Object Detection Network for Autonomous Driving