Proposal Flow: Semantic Correspondences from Object Proposals (2017-03-21T00:00:00.000000Z)

TL;DR

This work introduces a novel approach to semantic flow, dubbed proposal flow, that establishes reliable correspondences using object proposals, and demonstrates that proposal flow significantly outperforms existing semantic flow methods in various settings.

Abstract

Finding image correspondences remains a challenging problem in the presence of intra-class variations and large changes in scene layout. Semantic flow methods are designed to handle images depicting different instances of the same object or scene category. We introduce a novel approach to semantic flow, dubbed proposal flow, that establishes reliable correspondences using object proposals. Unlike prevailing semantic flow approaches that operate on pixels or regularly sampled local regions, proposal flow benefits from the characteristics of modern object proposals, that exhibit high repeatability at multiple scales, and can take advantage of both local and geometric consistency constraints among proposals. We also show that the corresponding sparse proposal flow can effectively be transformed into a conventional dense flow field. We introduce two new challenging datasets that can be used to evaluate both general semantic flow techniques and region-based approaches such as proposal flow. We use these benchmarks to compare different matching algorithms, object proposals, and region features within proposal flow, to the state of the art in semantic flow. This comparison, along with experiments on standard datasets, demonstrates that proposal flow significantly outperforms existing semantic flow methods in various settings.

Authors

J. Ponce

11 papers

Minsu Cho

5 papers

C. Schmid

33 papers

TL;DR

Abstract

Authors

References87 items

SCNet: Learning Semantic Correspondence

FCSS: Fully Convolutional Self-Similarity for Dense Semantic Correspondence

Universal Correspondence Network

Joint Recovery of Dense Correspondence and Cosegmentation in Two Images

Beyond Local Search: Tracking Objects Everywhere with Instance-Specific Proposals

WarpNet: Weakly Supervised Matching for Single-View Reconstruction

Learning Dense Correspondence via 3D-Guided Cycle Consistency

Exploiting Semantic Information and Deep Matching for Optical Flow

Optical Flow with Semantic Segmentation and Localized Layers

Discriminative Learning of Deep Convolutional Feature Point Descriptors

Local Convolutional Features with Unsupervised Training for Image Retrieval

Proposal Flow

DeepMatching: Hierarchical Deformable Dense Matching

FlowWeb: Joint image set alignment by weaving consistent, pixel-wise correspondences

Generalized Deformable Spatial Pyramid: Geometry-preserving dense correspondence estimation

Robust image filtering using joint static and dynamic guidance

Matching bags of regions in RGBD images

MatchNet: Unifying feature and metric learning for patch-based matching

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

Multi-image Matching via Fast Alternating Minimization

Dense Semantic Correspondence Where Every Pixel is a Classifier

Contextual Action Recognition with R*CNN

Fast R-CNN

Learning to compare image patches via convolutional neural networks

BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation

What Makes for Effective Detection Proposals?

Unsupervised object discovery and localization in the wild: Part-based matching with bottom-up region proposals

Virtual view networks for object reconstruction

Do Convnets Learn Correspondence?

Edge Boxes: Locating Object Proposals from Edges

Jointly Optimizing 3D Model Fitting and Fine-Grained Classification

Dense Correspondences across Scenes and Scales

Multiscale Combinatorial Grouping

Co-localization in Real-World Images

DAISY Filter Flow: A Generalized Discrete Approach to Dense Correspondences

Multi-fold MIL Training for Weakly Supervised Object Localization

Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition

Detect What You Can: Detecting and Representing Objects Using Holistic Models and Body Parts

Beyond PASCAL: A benchmark for 3D object detection in the wild

Scale-Space SIFT flow

Prime Object Proposals with Randomized Prim's Algorithm

Learning Graphs to Match

Articulated Human Detection with Flexible Mixtures of Parts

Deformable Spatial Pyramid Matching for Fast Dense Correspondences

Unsupervised Joint Object Discovery and Segmentation in Internet Images

Selective Search for Object Recognition

ImageNet classification with deep convolutional neural networks

A Naturalistic Open Source Movie for Optical Flow Evaluation

Collection flow

On SIFTs and their scales

Progressive graph matching: Making a move of graphs via probabilistic voting

Nonparametric Scene Parsing via Label Transfer

A graph-matching kernel for object categorization

Semantic contours from inverse detectors

Non-rigid dense correspondence with applications for image enhancement

The Caltech-UCSD Birds-200-2011 Dataset

Fast cost-volume filtering for visual correspondence and beyond

SIFT Flow: Dense Correspondence across Scenes and Its Applications

RASL: Robust alignment by sparse and low-rank decomposition for linearly correlated images

Poselets: Body part detectors trained using 3D human pose annotations

Object detection using a max-margin Hough transform

ImageNet: A large-scale hierarchical image database

A discriminatively trained, multiscale, deformable part model

Robust statistics on Riemannian manifolds via the geometric median

Robust Object Detection with Interleaved Categorization and Segmentation

A Database and Evaluation Methodology for Optical Flow

Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories

One-shot learning of object categories

Data driven image models through continuous joint alignment

Histograms of oriented gradients for human detection

Distinctive Image Features from Scale-Invariant Keypoints

Robust wide-baseline stereo from maximally stable extremal regions

Computer Vision: A Modern Approach

Approximate Thin Plate Spline Mappings

"Determining optical flow": A Retrospective

A multiple-baseline stereo