FIERY: Future Instance Prediction in Bird’s-Eye View from Surround Monocular Cameras (2021-04-21T00:00:00.000000Z)

TL;DR

FIERY is a probabilistic future prediction model in bird’s-eye view from monocular cameras that predicts future instance segmentation and motion of dynamic agents that can be transformed into non-parametric future trajectories.

Abstract

Driving requires interacting with road agents and predicting their future behaviour in order to navigate safely. We present FIERY: a probabilistic future prediction model in bird’s-eye view from monocular cameras. Our model predicts future instance segmentation and motion of dynamic agents that can be transformed into non-parametric future trajectories. Our approach combines the perception, sensor fusion and prediction components of a traditional autonomous driving stack by estimating bird’s-eye-view prediction directly from surround RGB monocular camera inputs. FIERY learns to model the inherent stochastic nature of the future solely from camera driving data in an end-to-end manner, without relying on HD maps, and predicts multimodal future trajectories. We show that our model outperforms previous prediction baselines on the NuScenes and Lyft datasets. The code and trained models are available at https://github.com/wayveai/fiery.

Authors

Vijay Badrinarayanan

7 papers

R. Cipolla

17 papers

Jeffrey Hawke

2 papers

TL;DR

Abstract

Authors

References53 items

Enabling spatio-temporal aggregation in Birds-Eye-View Vehicle Estimation

LiRaNet: End-to-End Trajectory Prediction using Spatio-Temporal Radar Fusion

Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D

BEV-Seg: Bird's Eye View Semantic Segmentation Using Geometry and Semantic Point Cloud

FISHING Net: Future Inference of Semantic Heatmaps In Grids

MultiXNet: Multiclass Multistage Multimodal Motion Prediction

Video Panoptic Segmentation

VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized Representation

Predicting Semantic Map Representations From Images Using Pyramid Occupancy Networks

MotionNet: Joint Perception and Motion Prediction for Autonomous Driving Based on Bird’s Eye View Maps

Probabilistic Future Prediction for Video Scene Understanding

CoverNet: Multimodal Behavior Prediction Using Trajectory Sets

Multiple Futures Prediction

On the Benefit of Adversarial Training for Monocular Depth Estimation

SpAGNN: Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Data

MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction

Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation

Cross-View Semantic Segmentation for Sensing Surroundings

Rules of the Road: Predicting Driving Behavior With a Convolutional Model of Semantic Interactions

EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

Monocular Plan View Networks for Autonomous Driving

PRECOG: PREdiction Conditioned on Goals in Visual Multi-Agent Settings

nuScenes: A Multimodal Dataset for Autonomous Driving

Pseudo-LiDAR From Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving

Orthographic Feature Transform for Monocular 3D Object Detection

IntentNet: Learning to Predict Intention from Raw Sensor Data

Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks

Deep Continuous Fusion for Multi-sensor 3D Object Detection

Uncertainty-aware Short-term Motion Prediction of Traffic Actors for Autonomous Driving

Generative Adversarial Frontal View to Bird View Synthesis

Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net

Monocular Semantic Occupancy Grid Mapping With Convolutional Variational Encoder–Decoder Networks

PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation

Predicting Scene Parsing and Motion Dynamics in the Future

Stochastic Variational Video Prediction

Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics

DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents

Predicting Deeper into the Future of Semantic Segmentation

The Energy of Data

Learning in an Uncertain World: Representing Ambiguity Through Multiple Hypotheses

Social LSTM: Human Trajectory Prediction in Crowded Spaces

Level 5

A Survey of Motion Planning and Control Techniques for Self-Driving Urban Vehicles

Deep Residual Learning for Image Recognition

Delving Deeper into Convolutional Networks for Learning Video Representations

Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding

Spatial Transformer Networks

Detection, prediction, and avoidance of dynamic obstacles in urban environments

Inevitable collision states. A step towards safer robots?

Motion Planning in Dynamic Environments Using Velocity Obstacles

The Hungarian method for the assignment problem

Lyft level 5 perception dataset

, Kaahan Radia , Jianfei Chen , Dequan Wang , Ionel Gog , and Joseph E . Gonzalez

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names