Multi-dialect Arabic BERT for Country-level Dialect Identification (2020-07-10T00:00:00.000000Z)

TL;DR

This paper presents the experiments conducted, and the models developed by the competing team, Mawdoo3 AI, along the way to achieving the winning solution to subtask 1 of the Nuanced Arabic Dialect Identification (NADI) shared task.

Abstract

Arabic dialect identification is a complex problem for a number of inherent properties of the language itself. In this paper, we present the experiments conducted, and the models developed by our competing team, Mawdoo3 AI, along the way to achieving our winning solution to subtask 1 of the Nuanced Arabic Dialect Identification (NADI) shared task. The dialect identification subtask provides 21,000 country-level labeled tweets covering all 21 Arab countries. An unlabeled corpus of 10M tweets from the same domain is also presented by the competition organizers for optional use. Our winning solution itself came in the form of an ensemble of different training iterations of our pre-trained BERT model, which achieved a micro-averaged F1-score of 26.78% on the subtask at hand. We publicly release the pre-trained language model component of our winning solution under the name of Multi-dialect-Arabic-BERT model, for any interested researcher out there.

Authors

Ibraheem Tuffaha

3 papers

Haitham Seelawi

2 papers

Wael Farhan

2 papers

TL;DR

Abstract

Authors

References30 items

NADI 2020: The First Nuanced Arabic Dialect Identification Shared Task

KUISAIL at SemEval-2020 Task 12: BERT-CNN for Offensive Speech Identification in Social Media

5分で分かる!? 有名論文ナナメ読み：Jacob Devlin et al. : BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding

Mawdoo3 AI at MADAR Shared Task: Arabic Tweet Dialect Identification

The MADAR Shared Task on Arabic Fine-Grained Dialect Identification

Mawdoo3 AI at MADAR Shared Task: Arabic Fine-Grained Dialect Identification with Ensemble Learning

Team JUST at the MADAR Shared Task on Arabic Fine-Grained Dialect Identification

No Army, No Navy: BERT Semi-Supervised Learning of Arabic Dialects

XLNet: Generalized Autoregressive Pretraining for Language Understanding

Pre-trained Word Embeddings for Arabic Aspect-Based Sentiment Analysis of Airline Tweets

SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing

Fine-Grained Arabic Dialect Identification

Deep Models for Arabic Dialect Identification on Benchmarked Data

Character Level Convolutional Neural Network for Arabic Dialect Identification

The MADAR Arabic Dialect Corpus and Lexicon

Arap-Tweet: A Large Multi-Dialect Twitter Corpus for Gender, Age and Language Variety Identification

You Tweet What You Speak: A City-Level Dataset of Arabic Dialects

Attention is All you Need

Analysis and Optimization of fastText Linear Text Classifier

Enriching Word Vectors with Subword Information

Bag of Tricks for Efficient Text Classification

Improved Arabic Dialect Classification with Social Media Data

Adam: A Method for Stochastic Optimization

Arabic Dialect Identification

Sentence Level Dialect Identification in Arabic

Scalable Modified Kneser-Ney Language Model Estimation

Efficient Estimation of Word Representations in Vector Space

ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators.

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

AraVec: A set of Arabic Word Embedding Models for use in Arabic NLP

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names