Contrastive Language–Image Pre-training for the Italian Language (2021-08-19T00:00:00.000000Z)

TL;DR

The first CLIP model for the Italian Language (CLIP-Italian) is presented, trained on more than 1.4 million image-text pairs and results show that CLIP- Italian outperforms the multilingual CLip model on the tasks of image retrieval and zero-shot classification.

Abstract

CLIP (Contrastive Language-Image Pre-training) is a very recent multi-modal model that jointly learns representations of images and texts. The model is trained on a massive amount of English data and shows impressive performance on zero-shot classification tasks. Training the same model on a different language is not trivial, since data in other languages might be not enough and the model needs high-quality translations of the texts to guarantee a good performance. In this paper, we present the first CLIP model for the Italian Language (CLIP-Italian), trained on more than 1.4 million image-text pairs. Results show that CLIP-Italian outperforms the multilingual CLIP model on the tasks of image retrieval and zero-shot classification.

Authors

Gabriele Sarti

5 papers

Federico Bianchi

9 papers

Giuseppe Attanasio

3 papers

TL;DR

Abstract

Authors

References34 items

A visual–language foundation model for pathology image analysis using medical Twitter

Stable Bias: Evaluating Societal Representations in Diffusion Models

Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale

When and why vision-language models behave like bags-of-words, and what to do about it?

Expert-level detection of pathologies from unannotated chest X-ray images via self-supervised learning

American == White in Multimodal Language-and-Image AI

Contrastive language and vision learning of general fashion concepts

Multimodal datasets: misogyny, pornography, and malignant stereotypes

Are Gender-Neutral Queries Really Gender-Neutral? Mitigating Gender Bias in Image Search

On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜

WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning

Learning Transferable Visual Models From Natural Language Supervision

How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models

BERT Goes Shopping: Comparing Distributional Models for Product Representations

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients

“You Sound Just Like Your Father” Commercial Machine Translation Systems Include Stylistic Biases

Making Monolingual Sentence Embeddings Multilingual Using Knowledge Distillation

What the [MASK]? Making Sense of Language-Specific BERT Models

5分で分かる!? 有名論文ナナメ読み：Jacob Devlin et al. : BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding

Large scale datasets for Image and Video Captioning in Italian

Energy and Policy Considerations for Deep Learning in NLP

Conceptual Captions: A Cleaned, Hypernymed, Image Alt-text Dataset For Automatic Image Captioning

SGDR: Stochastic Gradient Descent with Warm Restarts

Microsoft COCO: Common Objects in Context

mCLIP: Multilingual CLIP via Cross-lingual Transfer

Large-scale domain-specific pretraining for biomedical vision-language processing

On the Gap between Adoption and Understanding in NLP

McMillanMajor, and Shmargaret Shmitchell

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Speech Recognition

pre-trained model, 14 the modeling and training code, 15

g8a9/clip-italian/reports/

It5: Large-scale text-to-text pretraining for italian language understanding and generation

Field of Study

Journal Information

Name

Volume