Wiki-40B: Multilingual Language Model Dataset
Published in
International Conference on Language Resources ...(2020)External Links:
TL
TL;DR
A new multilingual language model benchmark that is composed of 40+ languages spanning several scripts and linguistic families with around 40 billion characters is proposed, and the task of multilingual causal language modeling is introduced.