3260 papers • 126 benchmarks • 313 datasets
Cross-lingual transfer refers to transfer learning using data and models available for one language for which ample such resources are available (e.g., English) to solve tasks in another, commonly more low-resource, language.
(Image credit: Papersgraph)
These leaderboards are used to track progress in cross-lingual-transfer-5
Use these libraries to find cross-lingual-transfer-5 models and implementations
An architecture to learn joint multilingual sentence representations for 93 languages, belonging to more than 30 different families and written in 28 different scripts using a single BiLSTM encoder with a shared byte-pair encoding vocabulary for all languages, coupled with an auxiliary decoder and trained on publicly available parallel corpora.
It is shown that pretraining multilingual language models at scale leads to significant performance gains for a wide range of cross-lingual transfer tasks, and the possibility of multilingual modeling without sacrificing per-language performance is shown for the first time.
This work explores the limits of contrastive learning as a way to train unsupervised dense retrievers and shows that it leads to strong performance in various retrieval settings and can perform cross-lingual retrieval between different scripts, such as retrieving English documents from Arabic queries.
The Cross-lingual TRansfer Evaluation of Multilingual Encoders XTREME benchmark is introduced, a multi-task benchmark for evaluating the cross-lingually generalization capabilities of multilingual representations across 40 languages and 9 tasks.
A novel two-step attention architecture for the inflection decoder is presented and the crucial factors for success with cross-lingual transfer for morphological inflection: typological similarity and a common representation across languages are identified.
Overall, it is found that the similarity between the percentage of words that get split into subwords in the source and target data (the isplit word ratio difference/i) is the strongest predictor for model performance on target data.
An information-theoretic framework that formulates cross-lingual language model pre- training as maximizing mutual information between multilingual-multi-granularity texts is presented and a new pre-training task based on contrastive learning is proposed.
It is experimentally demonstrated that high capacity multilingual language models applied in a zero-shot (model-based cross-lingual transfer) setting consistently outperform data-basedCross-lingUAL transfer approaches.
This work proposes a novel end-to-end model that learns to align and predict slots in a multilingual NLU system and uses the corpus to explore various cross-lingual transfer methods focusing on the zero-shot setting and leveraging MT for language expansion.
Adding a benchmark result helps the community track progress.