TL
TL;DR
This work analyzes CLIP’s ability to perform zero-shot learning on various texture and material classification datasets, and its ability to represent compositional properties of texture such as red dots or yellow stripes on the Describable Texture in Detail dataset.
Abstract
. We investigate how well does CLIP understand texture in natural images described by natural language. To this end we analyze CLIP’s ability to: (1) perform zero-shot learning on various texture and material classification datasets; (2) represent compositional properties of texture such as red dots or yellow stripes on the Describable Texture in Detail ( DTD 2 ) dataset; and (3) aid fine-grained categorization of birds in photographs described by color and texture of their body parts.
References23 items
1
Hierarchical Text-Conditional Image Generation with CLIP Latents
2
Image Segmentation Using Text and Image Prompts
3
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing
4
Open-vocabulary Object Detection via Vision and Language Knowledge Distillation
5