FlowNet: Learning Optical Flow with Convolutional Networks (2015-04-26T00:00:00.000000Z)

TL;DR

This paper constructs CNNs which are capable of solving the optical flow estimation problem as a supervised learning task, and proposes and compares two architectures: a generic architecture and another one including a layer that correlates feature vectors at different image locations.

Abstract

Convolutional neural networks (CNNs) have recently been very successful in a variety of computer vision tasks, especially on those linked to recognition. Optical flow estimation has not been among the tasks CNNs succeeded at. In this paper we construct CNNs which are capable of solving the optical flow estimation problem as a supervised learning task. We propose and compare two architectures: a generic architecture and another one including a layer that correlates feature vectors at different image locations. Since existing ground truth data sets are not sufficiently large to train a CNN, we generate a large synthetic Flying Chairs dataset. We show that networks trained on this unrealistic data still generalize very well to existing datasets such as Sintel and KITTI, achieving competitive accuracy at frame rates of 5 to 10 fps.

Authors

T. Brox

22 papers

Vladimir Golkov

2 papers

D. Cremers

21 papers

TL;DR

Abstract

Authors

References37 items

Optical Flow with Geometric Occlusion Estimation and Fusion of Multiple Frames

EpicFlow: Edge-preserving interpolation of correspondences for optical flow

Adam: A Method for Stochastic Optimization

Learning to generate chairs with convolutional neural networks

Hypercolumns for object segmentation and fine-grained localization

Fully convolutional networks for semantic segmentation

Fast Edge-Preserving PatchMatch for Large Displacement Optical Flow

Computing the stereo matching cost with a convolutional neural network

N4-Fields: Neural Network Nearest Neighbor Fields for Image Transforms

Seeing 3D Chairs: Exemplar Part-Based 2D-3D Alignment Using a Large Dataset of CAD Models

Caffe: Convolutional Architecture for Fast Feature Embedding

Depth Map Prediction from a Single Image using a Multi-Scale Deep Network

Descriptor Matching with Convolutional Neural Networks: a Comparison to SIFT

Unsupervised learning of depth and motion

Learning the Local Statistics of Optical Flow

Locally Affine Sparse-to-Dense Matching for Motion and Occlusion Estimation

DeepFlow: Large Displacement Optical Flow with Deep Matching

Visualizing and Understanding Convolutional Networks

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation

Vision meets robotics: The KITTI dataset

Learning Hierarchical Features for Scene Labeling

Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images

ImageNet classification with deep convolutional neural networks

A Naturalistic Open Source Movie for Optical Flow Evaluation

Efficient Closed-Form Solution to Generalized Boundary Detection

Adaptive deconvolutional networks for mid and high level feature learning

Convolutional Learning of Spatio-temporal Features

Learning Optical Flow

IM2GPS: estimating geographic information from a single image

A Database and Evaluation Methodology for Optical Flow

High Accuracy Optical Flow Estimation Based on a Theory for Warping

Dense estimation and object-based segmentation of the optical flow with robust techniques

Learning parameterized models of image motion

Backpropagation Applied to Handwritten Zip Code Recognition

Determining Optical Flow

Ieee Transactions on Pattern Analysis and Machine Intelligence 1 Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation

Bonn University

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names