Pushing the Limits of Low-Resource Morphological Inflection (2019-08-16T00:00:00.000000Z)

TL;DR

A novel two-step attention architecture for the inflection decoder is presented and the crucial factors for success with cross-lingual transfer for morphological inflection: typological similarity and a common representation across languages are identified.

Abstract

Recent years have seen exceptional strides in the task of automatic morphological inflection generation. However, for a long tail of languages the necessary resources are hard to come by, and state-of-the-art neural methods that work well under higher resource settings perform poorly in the face of a paucity of data. In response, we propose a battery of improvements that greatly improve performance under such low-resource conditions. First, we present a novel two-step attention architecture for the inflection decoder. In addition, we investigate the effects of cross-lingual transfer from single and multiple languages, as well as monolingual data hallucination. The macro-averaged accuracy of our models outperforms the state-of-the-art by 15 percentage points. Also, we identify the crucial factors for success with cross-lingual transfer for morphological inflection: typological similarity and a common representation across languages.

Authors

Graham Neubig

38 papers

Antonios Anastasopoulos

4 papers

TL;DR

Abstract

Authors

References39 items

The SIGMORPHON 2019 Shared Task: Morphological Analysis in Context and Cross-Lingual Transfer for Inflection

Exact Hard Monotonic Attention for Character-Level Transduction

Bootstrapping a Neural Morphological Analyzer for St. Lawrence Island Yupik from a Finite-State Transducer

Automatic Keyboard Layout Design for Low-Resource Latin-Script Languages

The CoNLL–SIGMORPHON 2018 Shared Task: Universal Morphological Reinflection

BME-HAS System for CoNLL–SIGMORPHON 2018 Shared Task: Universal Morphological Reinflection

Rapid Adaptation of Neural Machine Translation to New Languages

UniMorph 2.0: Universal Morphology

Leveraging translations for speech transcription in low-resource settings

Unsupervised Machine Translation Using Monolingual Corpora Only

Morphological Inflection Generation with Multi-space Variational Encoder-Decoders

Data Augmentation for Morphological Reinflection

Training Data Augmentation for Low-Resource Morphological Inflection

Attention is All you Need

CoNLL-SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection in 52 Languages

Controllable Invariance through Adversarial Feature Learning

URIEL and lang2vec: Representing languages as typological, geographical, and phylogenetic vectors

One-Shot Neural Cross-Lingual Transfer for Paradigm Completion

DyNet: The Dynamic Neural Network Toolkit

Morphological Inflection Generation with Hard Monotonic Attention

The SIGMORPHON 2016 Shared Task—Morphological Reinflection

Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classification

From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification

Incorporating Structural Alignment Biases into an Attentional Neural Translation Model

Multi-Source Neural Translation

Effective Approaches to Attention-based Neural Machine Translation

Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks

Domain-Adversarial Training of Neural Networks

Neural Machine Translation by Jointly Learning to Align and Translate

3. The verbal morphology of Maltese

IT–IST at the SIGMORPHON 2019 Shared Task: Sparse Two-headed Models for Inflection

Building Speech Recognition Systems for Language Documentation: The CoEDL Endangered Language Pipeline and Inference System (Elpis)

Improving Sequence to Sequence Learning for Morphological Inflection Generation: The BIU-MIT Systems for the SIGMORPHON 2016 Shared Task for Morphological Reinflection

Stanford Neural Machine Translation Systems for Spoken Language Domains

Indonesian: A Comprehensive Grammar

The verbal morphology of Maltese. Language Acquisition and Language Disorders

A Hyperparameters Here we list all the hyperparameters of our models

Here we list the optimizer settings: • Optimizer: SimpleSGD • Starting learning rate: 0.1 • Learning Rate Decay: 0.5 • Learning Rate Decay Patience: 6 epochs • Maximum number of epochs: 20

Ryan Cotterell, and Mans Hulden. 2019a. The SIGMORPHON 2019 shared task: Crosslinguality and context in morphology

Field of Study

Venue Information

Name

Type

URL

Alternate Names