IGLUE Dataset

About this Dataset

The Image-Grounded Language Understanding Evaluation (IGLUE) benchmark brings together—by both aggregating pre-existing datasets and creating new ones—visual question answering, cross-modal retrieval, grounded reasoning, and grounded entailment tasks across 20 diverse languages. The benchmark enables the evaluation of multilingual multimodal models for transfer learning, not only in a zero-shot setting, but also in newly defined few-shot learning setups.

Source: IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languages

Dataset Variants

MaRVLIGLUEXVNLIxGQAxFlickr&COWIT (IGLUE)

Papers1

IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languages

The Image-Grounded Language Understanding Evaluation benchmark enables the evaluation of multilingual multimodal models for transfer learning, not only in a zero-shot setting, but also in newly defined few-shot learning setups.

Tasks

EDIT

Visual Reasoning Zero-Shot Cross-Lingual Transfer Zero-Shot Cross-Lingual Text-to-Image Retrieval Zero-Shot Cross-Lingual Image-to-Text Retrieval Zero-Shot Cross-Lingual Visual Natural Language Inference Zero-Shot Cross-Lingual Visual Reasoning Zero-Shot Cross-Lingual Visual Question Answering Max-Shot Cross-Lingual Visual Natural Language Inference Max-Shot Cross-Lingual Visual Question Answering Max-Shot Cross-Lingual Visual Reasoning Max-Shot Cross-Lingual Text-to-Image Retrieval Max-Shot Cross-Lingual Image-to-Text Retrieval

Similar Datasets

IGLUE

IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languages

MNIST

CelebA

GLUE

IGLUE

IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languages

MNIST

CelebA

GLUE