PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume (2017-09-07T00:00:00.000000Z)

TL;DR

PWC-Net has been designed according to simple and well-established principles: pyramidal processing, warping, and the use of a cost volume, and outperforms all published optical flow methods on the MPI Sintel final pass and KITTI 2015 benchmarks.

Abstract

We present a compact but effective CNN model for optical flow, called PWC-Net. PWC-Net has been designed according to simple and well-established principles: pyramidal processing, warping, and the use of a cost volume. Cast in a learnable feature pyramid, PWC-Net uses the current optical flow estimate to warp the CNN features of the second image. It then uses the warped features and features of the first image to construct a cost volume, which is processed by a CNN to estimate the optical flow. PWC-Net is 17 times smaller in size and easier to train than the recent FlowNet2 model. Moreover, it outperforms all published optical flow methods on the MPI Sintel final pass and KITTI 2015 benchmarks, running at about 35 fps on Sintel resolution (1024 Ã— 436) images. Our models are available on our project website.

Authors

Ming-Yu Liu

13 papers

Jan Kautz

9 papers

Xiaodong Yang

12 papers

TL;DR

Abstract

Authors

References63 items

Bounding Boxes, Segmentations and Object Coordinates: How Important is Recognition for 3D Scene Flow Estimation in Autonomous Driving Scenarios?

MirrorFlow: Exploiting Symmetries in Joint Optical Flow and Occlusion Estimation

Optical Flow in Mostly Rigid Scenes

Accurate Optical Flow via Direct Cost Volume Processing

Computer Vision for Autonomous Vehicles: Problems, Datasets and State-of-the-Art

FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks

InterpoNet, a Brain Inspired Neural Network for Optical Flow Dense Interpolation

The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation

Optical Flow Estimation Using a Spatial Pyramid Network

Densely Connected Convolutional Networks

Back to Basics: Unsupervised Learning of Optical Flow via Brightness Constancy and Motion Smoothness

CNN-Based Patch Matching for Optical Flow with Thresholded Hinge Embedding Loss

DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

Efficient Coarse-to-Fine Patch Match for Large Displacement Optical Flow

Full Flow: Optical Flow Estimation By Global Optimization over Regular Grids

Exploiting Semantic Information and Deep Matching for Optical Flow

Learning Image Matching by Simply Watching Video

A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation

PatchBatch: A Batch Augmented Loss for Optical Flow

Multi-Scale Context Aggregation by Dilated Convolutions

Blind video temporal consistency

Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches

Flow Fields: Dense Correspondence Fields for Highly Accurate Large Displacement Optical Flow Estimation

Efficient sparse-to-dense optical flow estimation using a learned basis and layers

Object scene flow for autonomous vehicles

Spatial Transformer Networks

U-Net: Convolutional Networks for Biomedical Image Segmentation

FlowNet: Learning Optical Flow with Convolutional Networks

EpicFlow: Edge-preserving interpolation of correspondences for optical flow

Caffe: Convolutional Architecture for Fast Feature Embedding

Two-Stream Convolutional Networks for Action Recognition in Videos

A Quantitative Analysis of Current Practices in Optical Flow Estimation and the Principles Behind Them

ImageNet classification with deep convolutional neural networks

A Naturalistic Open Source Movie for Optical Flow Evaluation

Are we ready for autonomous driving? The KITTI vision benchmark suite

Fast cost-volume filtering for visual correspondence and beyond

A generative perspective on MRFs in low-level vision

An Improved Algorithm for TV-L 1 Optical Flow

Et al

Learning for Optical Flow Using Stochastic Optimization

Learning Optical Flow

Extracting and composing robust features with denoising autoencoders

Human-assisted motion annotation

A Database and Evaluation Methodology for Optical Flow

Unsupervised Learning of Image Transformations

Bilateral Filtering-Based Optical Flow Estimation with Occlusion Detection

On the Spatial Statistics of Optical Flow

High Accuracy Optical Flow Estimation Based on a Theory for Warping

A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms

Robust computation of optical flow in a multi-scale differential framework

Performance of optical flow techniques

Probability distributions of optical flow

Backpropagation Applied to Handwritten Zip Code Recognition

Determining Optical Flow

Semi-Supervised Learning for Optical Flow with Generative Adversarial Networks

Author manuscript, published in "IEEE Intenational Conference on Computer Vision (ICCV), Sydney: Australie (2013)" DeepFlow: Large displacement optical flow with deep matching

Anisotropic Huber-L1 Optical Flow

The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields

AND T

Ieee Transactions on Pattern Analysis and Machine Intelligence 1 Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation

Ieee Transactions on Pattern Analysis and Machine Intelligence 1 Motion Detail Preserving Optical Flow Estimation

Universität Des Saarlandes Fachrichtung 6.1 – Mathematik Lucas/kanade Meets Horn/schunck: Combining Local and Global Optic Flow Methods Lucas/kanade Meets Horn/schunck: Combining Local and Global Optic Flow Methods

c ○ 2000 Kluwer Academic Publishers. Manufactured in The Netherlands. Learning Low-Level Vision

Field of Study

Journal Information

Name

Page