A Neural Architecture for Dialectal Arabic Segmentation (2017-04-01T00:00:00.000000Z)

TL;DR

This paper shows how a segmenter can be trained using only 350 annotated tweets using neural networks without any normalization or use of lexical features or lexical resources.

Abstract

The automated processing of Arabic Dialects is challenging due to the lack of spelling standards and to the scarcity of annotated data and resources in general. Segmentation of words into its constituent parts is an important processing building block. In this paper, we show how a segmenter can be trained using only 350 annotated tweets using neural networks without any normalization or use of lexical features or lexical resources. We deal with segmentation as a sequence labeling problem at the character level. We show experimentally that our model can rival state-of-the-art methods that rely on additional resources.

Authors

Hamdy Mubarak

7 papers

Kareem Darwish

3 papers

Younes Samih

1 papers

TL;DR

Abstract

Authors

References37 items

QCRI $@$ DSL 2016: Spoken Arabic Dialect Identification Using Textual Features

Multilingual Code-switching Identification via LSTM Recurrent Neural Networks

Multi-view Dimensionality Reduction for Dialect Identification of Arabic Broadcast Speech

Farasa: A Fast and Furious Segmenter for Arabic

End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF

Neural Architectures for Named Entity Recognition

Bi-directional LSTM Recurrent Neural Network for Chinese Word Segmentation

Neural Machine Translation of Rare Words with Subword Units

Character-Aware Neural Language Models

Bidirectional LSTM-CRF Models for Sequence Tagging

A Critical Review of Recurrent Neural Networks for Sequence Learning

Boosting Named Entity Recognition with Neural Character Embeddings

Verifiably Effective Arabic Dialect Identification

Word Segmentation of Informal Arabic with Domain Adaptation

MADAMIRA: A Fast, Comprehensive Tool for Morphological Analysis and Disambiguation of Arabic

Arabic Dialect Identification

Hybrid speech recognition with Deep Bidirectional LSTM

Generating Sequences With Recurrent Neural Networks

Speech recognition with deep recurrent neural networks

Improving neural networks by preventing co-adaptation of feature detectors

Machine Translation of Arabic Dialects

Annotating and Learning Morphological Segmentation of Egyptian Colloquial Arabic

Natural Language Processing (Almost) from Scratch

Spoken Arabic Dialect Identification Using Phonotactic Modeling

2005 Special Issue: Framewise phoneme classification with bidirectional LSTM and other neural network architectures

Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data

Bidirectional recurrent neural networks

Long Short-Term Memory

Learning long-term dependencies with gradient descent is difficult

keras

A Multidialectal Parallel Corpus of Arabic

Developing an Egyptian Arabic Treebank: Impact of Dialectal Morphology on Annotation and Tool Development

Morphological Analysis and Disambiguation for Dialectal Arabic

Translating Dialectal Arabic to English

Conventional Orthography for Dialectal Arabic

Borrowing in modern standard arabic. Innovation and Continuity in Language and Communication of Different Language Cultures 9

Overfitting in Neural Nets: Backpropagation, Conjugate Gradient, and Early Stopping

Field of Study