natural-language-processing-11

Pretrained Multilingual Language Models

3260 papers • 126 benchmarks • 313 datasets

This task has no description! Would you like to contribute one?

(Image credit: Papersgraph)

Benchmarks

These leaderboards are used to track progress in pretrained-multilingual-language-models-11

Trend

Dataset

Best Model

Actions

No benchmarks available.

Libraries

i

Use these libraries to find pretrained-multilingual-language-models-11 models and implementations

Datasets

Belebele

Subtasks

No subtasks available.

Most implemented papers

How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models

Sebastian Ruder, Ivan Vulic, Iryna Gurevych, Jonas Pfeiffer, Phillip Rust•Wed Dec 30 2020

It is found that replacing the original multilingual tokenizer with the specialized monolingual tokenizer improves the downstream performance of the multilingual model for almost every task and language.

331

Content

0

Paper Graph

Investigating Math Word Problems using Pretrained Multilingual Language Models

Lingxiao Jiang, Minghuan Tan, Lei Wang, Jing Jiang•Tue May 18 2021

The experiments show that the MWP solvers may not be transferred to a different language even if the target expressions share the same numerical constants and operator set, and it can be better generalized if problem types exist on both source language and target language.

36 0

Paper Graph

Specializing Multilingual Language Models: An Empirical Study

Noah A. Smith, Ethan C. Chau•Tue Jun 15 2021

These evaluations on part-of-speech tagging, universal dependency parsing, and named entity recognition in nine diverse low-resource languages uphold the viability of these approaches while raising new questions around how to optimally adapt multilingual models to low- resource settings.

30 0

Paper Graph

Improving Word Translation via Two-Stage Contrastive Learning

A. Korhonen, Ivan Vulic, Nigel Collier, Fangyu Liu, Yaoyiran Li•Mon Mar 14 2022

This work proposes a robust and effective two-stage contrastive learning framework for the BLI task, and proposes to refine standard cross-lingual linear maps between static word embeddings (WEs) via a Contrastive learning objective.

31 0

Paper Graph

To Adapt or to Fine-tune: A Case Study on Abstractive Summarization

Pinzhen Chen, Zheng Zhao•Mon Aug 29 2022

In this work, multifaceted investigations on fine-tuning and adapters for summarization tasks with varying complexity: language, domain, and task transfer provide insights on multilinguality, model convergence, and robustness, hoping to shed light on the pragmatic choice of fine- tuning or adapters in abstractive summarization.

3 0

Paper Graph

Robustification of Multilingual Language Models to Real-world Noise in Crosslingual Zero-shot Settings with Robust Contrastive Pretraining

He He, Asa Cooper Stickland, Saab Mansour, Sailik Sengupta, Jason Krone•Sun Oct 09 2022

After investigating several ways to boost the robustness of multilingual models in this setting, this work proposes Robust Contrastive Pretraining (RCP), which combines data augmentation with a contrastive loss term at the pretraining stage and achieves large improvements on noisy data.

11 0

Paper Graph

Are Pretrained Multilingual Models Equally Fair across Languages?

Anders Søgaard, Laura Cabello Piqueras•Mon Oct 10 2022

This work evaluates three multilingual models on MozArt –mBERT, XLM-R, and mT5– and shows that across the four target languages, the three models exhibit different levels of group disparity, e.g., exhibiting near-equal risk for Spanish, but high levels of disparity for German.

10 0

Paper Graph

Language Agnostic Multilingual Information Retrieval with Contrastive Learning

Zhiheng Huang, Peng Qi, Kun Liu, William Yang Wang, Xiyang Hu, Xinchi Chen, Deguang Kong•Tue Oct 11 2022

An effective method to train multilingual IR systems when only English IR training data and some parallel corpora between English and other languages are available is presented and a semantic contrastive loss to align representations of parallel sentences that share the same semantics in different languages is designed.

12 0

Paper Graph

Improving Bilingual Lexicon Induction with Cross-Encoder Reranking

A. Korhonen, Ivan Vulic, Fangyu Liu, Yaoyiran Li•Sat Oct 29 2022

This work proposes a novel semi-supervised post-hoc reranking method termed BLICEr (BLI with Cross-Encoder Reranking), applicable to any precalculated CLWE space, which improves their BLI capability and substantially outperforms a series of strong baselines across the board.

12 0

Paper Graph

Adding a benchmark result helps the community track progress.