ILIAS is a large-scale test dataset for evaluation on Instance-Level Image retrieval At Scale. It is designed to support future research in image-to-image and text-to-image retrieval for particular objects and serves as a benchmark for evaluating representations of foundation or customized vision and vision-language models, as well as specialized retrieval techniques.
website | dataset | arxiv | huggingface
Composition
The dataset includes 1,000 object instances across diverse domains, with:
- 5,947 images in total:
- 1,232 image queries, depicting query objects on clean or uniform background
- 4,715 positive images, featuring the query objects in real-world conditions with clutter, occlusions, scale variations, and partial views
- 1,000 text queries, providing fine-grained textual descriptions of the query objects
- 100M distractors from YFCC100M to evaluate retrieval performance under large-scale settings, while asserting noise-free ground truth
