Learning from All Vehicles (2022-03-22T00:00:00.000000Z)

TL;DR

A system to train driving policies from experiences collected not just from the ego-vehicle, but all vehicles that it observes, which outperforms all prior methods on the public CARLA Leaderboard by a wide margin.

Abstract

In this paper, we present a system to train driving policies from experiences collected not just from the ego-vehicle, but all vehicles that it observes. This system uses the behaviors of other agents to create more diverse driving scenarios without collecting additional data. The main difficulty in learning from other vehicles is that there is no sensor information. We use a set of supervisory tasks to learn an intermediate representation that is invariant to the viewpoint of the controlling vehicle. This not only provides a richer signal at training time but also allows more complex reasoning during inference. Learning how all vehicles drive helps predict their behavior at test time and can avoid collisions. We evaluate this system in closed-loop driving simulations. Our system outperforms all prior methods on the public CARLA Leaderboard by a wide margin, improving driving score by 25 and route completion rate by 24 points.

Authors

Philipp Krähenbühl

10 papers

Dian Chen

1 papers

TL;DR

Abstract

Authors

References55 items

GRI: General Reinforced Imitation and its Application to Vision-Based Autonomous Driving

Learning to Simulate Self-Driven Particles System with Coordinated Policy Optimization

PredictionNet: Real-Time Joint Probabilistic Traffic Prediction for Planning, Control, and Simulation

NEAT: Neural Attention Fields for End-to-End Autonomous Driving

HDMapNet: An Online HD Map Construction and Evaluation Framework

Learning by Watching

Learning to drive from a world on rails

FIERY: Future Instance Prediction in Bird’s-Eye View from Surround Monocular Cameras

Multi-Modal Fusion Transformer for End-to-End Autonomous Driving

PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learning

MP3: A Unified Model to Map, Perceive, Predict and Plan

TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors

LookOut: Diverse Multi-Future Prediction and Planning for Self-Driving

StrObe: Streaming Object Detection from LiDAR Packets

TNT: Target-driveN Trajectory Prediction

Center-based 3D Object Detection and Tracking

VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized Representation

Tracking Objects as Points

Gen-LaneNet: A Generalized and Scalable Approach for 3D Lane Detection

Learning by Cheating

Scalability in Perception for Autonomous Driving: Waymo Open Dataset

End-to-End Model-Free Reinforcement Learning for Urban Driving Using Implicit Affordances

PointPainting: Sequential Fusion for 3D Object Detection

End-to-End Multi-View Fusion for 3D Object Detection in LiDAR Point Clouds

MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction

Exploiting Sparse Semantic HD Maps for Self-Driving Vehicle Localization

End-To-End Interpretable Neural Motion Planner

Objects as Points

PointPillars: Fast Encoders for Object Detection From Point Clouds

ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst

3D-LaneNet: End-to-End 3D Multiple Lane Detection

Microscopic Traffic Simulation using SUMO

IntentNet: Learning to Predict Intention from Raw Sensor Data

Deep Continuous Fusion for Multi-sensor 3D Object Detection

Learning to Drive in a Day

Conditional Affordance Learning for Driving in Urban Environments

Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net

CARLA: An Open Urban Driving Simulator

End-to-End Driving Via Conditional Imitation Learning

Rates of Motor Vehicle Crashes, Injuries and Deaths in Relation to Driver Age, United States, 2014-2015

High definition map-based vehicle localization for highly automated driving: Geometric analysis

DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents

DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

A perception‐driven autonomous urban vehicle

Odin: Team VictorTango's entry in the DARPA Urban Challenge

Autonomous driving in urban environments: Boss and the Urban Challenge

Abbas Sadat, Bin Yang, Sergio Casas, and Raquel Urtasun. Endto-end interpretable neural motion planner

ERFNet: Efficient Residual Factorized ConvNet for Real-Time Semantic Segmentation

ALVINN, an autonomous land vehicle in a neural network

Odin: Team VictorTango's Entry in the DARPA Urban Challenge

Practical Search Techniques in Path Planning for Autonomous Driving

Map-Based Precision Vehicle Localization in Urban Environments

Driving

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names