Label Propagation for Zero-shot Classification with Vision-Language Models (2024-04-05T00:00:00.000000Z)

TL;DR

The case of zero-shot classification in the presence of unlabeled data is tackled and ZLaP, a method based on label propagation (LP) that utilizes geodesic distances for classification is introduced, a method based on label propagation that utilizes geodesic distances for classification.

Abstract

Vision-Language Models (VLMs) have demonstrated im-pressive performance on zero-shot classification, i.e. classi-fication when provided merely with a list of class names. In this paper, we tackle the case of zero-shot classification in the presence of unlabeled data. We leverage the graph structure of the unlabeled data and introduce ZLaP, a method based on label propagation (LP) that utilizes geodesic distances for classification. We tailor LP to graphs containing both text and image features and further pro-pose an efficient method for performing inductive infer-ence based on a dual solution and a sparsification step. We perform extensive experiments to evaluate the effectiveness of our method on 14 common datasets and show that ZLaP outperforms the latest related works. Code: https://github.com/vladan-stojnic/ZLaP

Authors

Giorgos Tolias

11 papers

Yannis Kalantidis

9 papers

Vladan Stojni'c

1 papers

TL;DR

Abstract

Authors

References47 items

EVA-CLIP-18B: Scaling CLIP to 18 Billion Parameters

Intra-Modal Proxy Learning for Zero-Shot Visual Categorization with CLIP

ReCLIP: Refine Contrastive Language Image Pre-Training with Source Free Domain Adaptation

Neural Priming for Sample-Efficient Adaptation

Waffling around for Performance: Visual Classification with Random Words and Broad Concepts

Not All Features Matter: Enhancing Few-shot CLIP with Adaptive Prior Refinement

Prompt, Generate, Then Cache: Cascade of Foundation Models Makes Strong Few-Shot Learners

Test-Time Distribution Normalization for Contrastively Learned Visual-language Models

CHiLS: Zero-Shot Image Classification with Hierarchical Label Sets

Reproducible Scaling Laws for Contrastive Language-Image Learning

SuS-X: Training-Free Name-Only Transfer of Vision-Language Models

LAION-5B: An open large-scale dataset for training next generation image-text models

Is synthetic data from generative models ready for image recognition?

Visual Classification via Description from Large Language Models

SVL-Adapter: Self-Supervised Adapter for Vision-Language Pretrained Models

CALIP: Zero-Shot Enhancement of CLIP with Parameter-free Attention

Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models

What does a platypus look like? Generating customized prompts for zero-shot image classification

Unsupervised Prompt Learning for Vision-Language Models

Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning

BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation

Learning to Prompt for Vision-Language Models

Align before Fuse: Vision and Language Representation Learning with Momentum Distillation

Learning Transferable Visual Models From Natural Language Supervision

Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision

Label Propagation for Deep Semi-Supervised Learning

EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification

Fast Spectral Ranking for Similarity Search

Efficient Diffusion on Region Manifolds: Recovering Small Objects with Compact CNN Representations

Fast, Exact and Multi-scale Inference for Semantic Image Segmentation with Deep Gaussian CRFs

Food-101 - Mining Discriminative Components with Random Forests

ImageNet Large Scale Visual Recognition Challenge

Microsoft COCO: Common Objects in Context

3D Object Representations for Fine-Grained Categorization

Describing Textures in the Wild

Fine-Grained Visual Classification of Aircraft

UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild

SUN database: Large-scale scene recognition from abbey to zoo

Automated Flower Classification over a Large Number of Classes

Random Walks for Image Segmentation

Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories

Learning with Local and Global Consistency

Care of aged doctors

Learning Multiple Layers of Features from Tiny Images

Visualizing Data using t-SNE

Label Propagation and Quadratic Criterion

licensed use limited to the terms of the

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names