Optical Flow Estimation Using a Spatial Pyramid Network
TL
TL;DR
The Spatial Pyramid Network (SPyNet) is much simpler and 96% smaller than FlowNet in terms of model parameters, which makes it more efficient and appropriate for embedded applications.
Abstract
We learn to compute optical flow by combining a classical spatial-pyramid formulation with deep learning. This estimates large motions in a coarse-to-fine approach by warping one image of a pair at each pyramid level by the current flow estimate and computing an update to the flow. Instead of the standard minimization of an objective function at each pyramid level, we train one deep network per level to compute the flow update. Unlike the recent FlowNet approach, the networks do not need to deal with large motions, these are dealt with by the pyramid. This has several advantages. First, our Spatial Pyramid Network (SPyNet) is much simpler and 96% smaller than FlowNet in terms of model parameters. This makes it more efficient and appropriate for embedded applications. Second, since the flow at each pyramid level is small (
References55 items
1
Deep Discrete Flow
2
Scalable Robust Principal Component Analysis using Grassmann Averages.
3
Back to Basics: Unsupervised Learning of Optical Flow via Brightness Constancy and Motion Smoothness
4
Fast Optical Flow Using Dense Inverse Search
5