Lightweight Adapter Tuning for Multilingual Speech Translation (2021-06-02T00:00:00.000000Z)

TL;DR

This paper proposes a comprehensive analysis of adapters for multilingual speech translation (ST) and shows that adapters can be used to efficiently specialize ST to specific language pairs with a low extra cost in terms of parameters.

Abstract

Adapter modules were recently introduced as an efficient alternative to fine-tuning in NLP. Adapter tuning consists in freezing pre-trained parameters of a model and injecting lightweight modules between layers, resulting in the addition of only a small number of task-specific trainable parameters. While adapter tuning was investigated for multilingual neural machine translation, this paper proposes a comprehensive analysis of adapters for multilingual speech translation (ST). Starting from different pre-trained models (a multilingual ST trained on parallel data or a multilingual BART (mBART) trained on non parallel multilingual data), we show that adapters can be used to: (a) efficiently specialize ST to specific language pairs with a low extra cost in terms of parameters, and (b) transfer from an automatic speech recognition (ASR) task and an mBART pre-trained model to a multilingual ST task. Experiments show that adapter tuning offer competitive results to full fine-tuning, while being much more parameter-efficient.

Authors

Jiatao Gu

6 papers

Changhan Wang

8 papers

J. Pino

8 papers

TL;DR

Abstract

Authors

References42 items

End-to-End Speech Translation with Pre-trained Models and Adapters: UPC at IWSLT 2021

Adapt-and-Adjust: Overcoming the Long-Tail Problem of Multilingual Speech Recognition

Language Adapters for Zero Shot Neural Machine Translation

Enabling Zero-Shot Multilingual Spoken Language Translation with Language-Specific Encoders and Decoders

Dual-decoder Transformer for Joint Automatic Speech Recognition and Multilingual Speech Translation

Multilingual Speech Recognition with Self-Attention Structured Parameterization

Fairseq S2T: Fast Speech-to-Text Modeling with Fairseq

Transformers: State-of-the-Art Natural Language Processing

Adaptable Multi-Domain Language Model for Transformer ASR

Multilingual Translation with Extensible Multilingual Pretraining and Finetuning

AdapterHub: A Framework for Adapting Transformers

wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations

AdapterFusion: Non-Destructive Task Composition for Transfer Learning

Recipes for Adapting Pre-trained Monolingual and Multilingual Models to Machine Translation

MAD-X: An Adapter-based Framework for Multi-task Cross-lingual Transfer

ESPnet-ST: All-in-One Speech Translation Toolkit

Multilingual Denoising Pre-training for Neural Machine Translation

BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

Simple, Scalable Adaptation for Neural Machine Translation

Large-Scale Multilingual Speech Recognition with a Streaming End-to-End Model

MuST-C: a Multilingual Speech Translation Corpus

SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition

Parameter-Efficient Transfer Learning for NLP

Pre-training on high-resource speech recognition improves low-resource speech-to-text translation

SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing

Attention is All you Need

Learning multiple visual domains with residual adapters

Layer Normalization

Adam: A Method for Stochastic Optimization

Statistical Significance Tests for Machine Translation Evaluation

Backpropagation Applied to Handwritten Zip Code Recognition

Multilingual Speech Translation from Efficient Finetuning of Pretrained Models

Bert and pals: Projected attention layers for efficient adaptation in multi-task learning

Neocognitron: A Self-Organizing Neural Network Model for a Mechanism of Visual Pattern Recognition

performs operations in parallel with a full layer produces the best result

A Parallel Adapters In this section, we present our preliminary experiments in which we explore different positions

Table 5: BLEU on dev set for parallel vs. serial adapters

Adapter # params (M) d h type trainable/total en-de

128 - ser 0.4/32.5 22. 3 128 4 par-TL 0.8/32.9 21. 4 128 4 par-SA 0.8/32.9 19. 5 128 4 par-XA 0.8/32.9

Specializing In addition to the reﬁnement recipe where language-speciﬁc adapters tailor the frozen multilingual

Trans-former layers or their sub-layers. We perform experiments where the adapters are added to the de-coder

the position of the parallel adapters: with the Transformer layer ( TL ), or with self-attention sub-layer ( SA ), or with cross-attention sub-layer Among the parallel variants, the one

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names