Localizing Objects with Self-Supervised Transformers and no Labels (2021-09-29T00:00:00.000000Z)

TL;DR

This work proposes a simple approach, LOST, that leverages the activation features of a vision transformer pre-trained in a self-supervised manner, that outperform state-of-the-art object discovery methods by up to 8 CorLoc points on PASCAL VOC 2012.

Abstract

Localizing objects in image collections without supervision can help to avoid expensive annotation campaigns. We propose a simple approach to this problem, that leverages the activation features of a vision transformer pre-trained in a self-supervised manner. Our method, LOST, does not require any external object proposal nor any exploration of the image collection; it operates on a single image. Yet, we outperform state-of-the-art object discovery methods by up to 8 CorLoc points on PASCAL VOC 2012. We also show that training a class-agnostic detector on the discovered objects boosts results by another 7 points. Moreover, we show promising results on the unsupervised object discovery task. The code to reproduce our results can be found at https://github.com/valeoai/LOST.

Authors

P. P'erez

4 papers

J. Ponce

11 papers

Huy V. Vo

5 papers

TL;DR

Abstract

Authors

References82 items

Revisiting the Calibration of Modern Neural Networks

Large-Scale Unsupervised Object Discovery

Scaling Vision Transformers

SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers

Intriguing Properties of Vision Transformers

Are Convolutional Neural Networks or Transformers more like human vision?

Segmenter: Transformer for Semantic Segmentation

Self-Supervised Learning with Swin Transformers

Emerging Properties in Self-Supervised Vision Transformers

Unsupervised Layered Image Decomposition into Object Prototypes

Self-supervised object detection from audio-visual correspondence

Going deeper with Image Transformers

Understanding Robustness of Transformers for Image Classification

Unbiased Teacher for Semi-Supervised Object Detection

Training Vision Transformers for Image Retrieval

Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet

Training data-efficient image transformers & distillation through attention

Unsupervised Object Detection with LiDAR Clues

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Deformable DETR: Deformable Transformers for End-to-End Object Detection

Self-Supervised Learning of Audio-Visual Objects from Video

Generative Pretraining From Pixels

Toward unsupervised, multi-object discovery in large-scale image collections

Object-Centric Learning with Slot Attention

Unsupervised Learning of Visual Features by Contrasting Cluster Assignments

Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning

End-to-End Object Detection with Transformers

Instance-Aware, Context-Focused, and Memory-Efficient Weakly Supervised Object Detection

Learning Representations by Predicting Bags of Visual Words

Proposal Learning for Semi-Supervised Object Detection

Momentum Contrast for Unsupervised Visual Representation Learning

Self-labelling via simultaneous clustering and representation learning

On the Relationship between Self-Attention and Convolutional Layers

Active Learning for Deep Detection Neural Networks

GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representations

Stand-Alone Self-Attention in Vision Models

Unsupervised Image Matching and Object Discovery as Optimization

Multi-Object Representation Learning with Iterative Variational Inference

Object Discovery From a Single Unlabeled Image by Mining Frequent Itemsets With Multi-Scale Features

MONet: Unsupervised Scene Decomposition and Representation

NOTE-RCNN: NOise Tolerant Ensemble RCNN for Semi-Supervised Object Detection

Invariant Information Clustering for Unsupervised Image Classification and Segmentation

PCL: Proposal Cluster Learning for Weakly Supervised Object Detection

Unsupervised Representation Learning by Predicting Image Rotations

Data Distillation: Towards Omni-Supervised Learning

Relation Networks for Object Detection

Non-local Neural Networks

Unsupervised Object Discovery and Co-Localization by Deep Descriptor Transforming

Attention is All you Need

Deep Self-Taught Learning for Weakly Supervised Object Localization

Mask R-CNN

Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization

CliqueCNN: Deep Unsupervised Exemplar Learning

Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles

Colorful Image Colorization

Deep Residual Learning for Image Recognition

Mining And-Or Graphs for Graph Matching and Object Discovery

Weakly Supervised Deep Detection Networks

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

Fast R-CNN

Unsupervised object discovery and localization in the wild: Part-based matching with bottom-up region proposals

Object Detectors Emerge in Deep Scene CNNs

Fully convolutional networks for semantic segmentation

Self-taught object localization with deep networks

Edge Boxes: Locating Object Proposals from Edges

Very Deep Convolutional Networks for Large-Scale Image Recognition

Microsoft COCO: Common Objects in Context

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation

Selective Search for Object Recognition

Unsupervised object class discovery via saliency-guided multiple class learning

Measuring the Objectness of Image Windows

Unsupervised Detection of Regions of Interest Using Iterative Link Analysis

ImageNet: A large-scale hierarchical image database

Non-negative matrix factorisation for object class discovery and image auto-annotation

Using Multiple Segmentations to Discover Objects and their Extent in Image Collections

Unsupervised Learning of Categories from Sets of Partially Matching Image Features