MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languages (2022-04-18T00:00:00.000000Z)

TL;DR

The MASSIVE dataset–Multilingual Amazon Slu resource package (SLURP) for Slot-filling, Intent classification, and Virtual assistant Evaluation is presented and modeling results on XLM-R and mT5 are presented, including exact match accuracy, intent classification accuracy, and slot- filling F1 score.

Abstract

We present the MASSIVE dataset–Multilingual Amazon Slu resource package (SLURP) for Slot-filling, Intent classification, and Virtual assistant Evaluation. MASSIVE contains 1M realistic, parallel, labeled virtual assistant utterances spanning 51 languages, 18 domains, 60 intents, and 55 slots. MASSIVE was created by tasking professional translators to localize the English-only SLURP dataset into 50 typologically diverse languages from 29 genera. We also present modeling results on XLM-R and mT5, including exact match accuracy, intent classification accuracy, and slot-filling F1 score. We have released our dataset, modeling code, and models publicly.

Authors

P. Natarajan

4 papers

Jack G. M. FitzGerald

1 papers

C. Hench

1 papers

TL;DR

Abstract

Authors

References68 items

Association of Sex With Neurobehavioral Markers of Executive Function in 2-Year-Olds at High and Low Likelihood of Autism

Alexa Teacher Model: Pretraining and Distilling Multi-Billion-Parameter Encoders for Natural Language Understanding Systems

Translate & Fill: Improving Zero-Shot Multilingual Semantic Parsing with Synthetic Data

The Flores-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation

Leveraging ParsBERT and Pretrained mT5 for Persian Abstractive Text Summarization

ParsiNLU: A Suite of Language Understanding Challenges for Persian

SLURP: A Spoken Language Understanding Resource Package

mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer

STIL - Simultaneous Slot Filling, Translation, Intent Classification, and Language Identification: Initial Results using mBART on MultiATIS++

MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmark

Pre-training via Paraphrasing

Low-resource Languages: A Review of Past Work and Future Challenges

Establishing Baselines for Text Classification in Low-Resource Languages

End-to-End Slot Alignment and Recognition for Cross-Lingual NLU

Low Resource Neural Machine Translation: A Benchmark for Five African Languages

XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization

Investigating an Approach for Low Resource Language Dataset Creation, Curation and Classification: Setswana and Sepedi

Don’t Parse, Generate! A Sequence to Sequence Architecture for Task-Oriented Semantic Parsing

Multilingual Denoising Pre-training for Neural Machine Translation

Cross-Lingual Ability of Multilingual BERT: An Empirical Study

Unsupervised Cross-lingual Representation Learning at Scale

On the Cross-lingual Transferability of Monolingual Representations

CATSLU: The 1st Chinese Audio-Textual Spoken Language Understanding Challenge

RoBERTa: A Robustly Optimized BERT Pretraining Approach

How Multilingual is Multilingual BERT?

Speech Model Pre-training for End-to-End Spoken Language Understanding

Benchmarking Natural Language Understanding Services for building Conversational Agents

BERT for Joint Intent Classification and Slot Filling

Cross-lingual Language Model Pretraining

Cross-lingual Transfer Learning for Multilingual Task Oriented Dialog

Spoken Language Understanding on the Edge

Semantic Parsing for Task Oriented Dialog using Hierarchical Representations

Multi-Source Cross-Lingual Model Transfer: Learning What to Share

Speaking

Zero-Shot Cross-lingual Classification Using Multilingual Neural Machine Translation

Neural Cross-Lingual Named Entity Recognition with Minimal Resources

Microsoft Dialogue Challenge: Building End-to-End Task-Completion Dialogue Systems

Tune: A Research Platform for Distributed Model Selection and Training

A Multi-task Approach to Learning Multilingual Representations

Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces

(Almost) Zero-Shot Cross-Lingual Spoken Language Understanding

Massively Parallel Hyperparameter Tuning

Cheap Translation for Cross-Lingual Named Entity Recognition

SQuAD: 100,000+ Questions for Machine Comprehension of Text

LORELEI Language Packs: Data, Tools, and Resources for Technology Development in Low Resource Languages

Adam: A Method for Stochastic Optimization

Links that speak: The global language network and its association with global fame

Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures

Parallel Data, Tools and Interfaces in OPUS

Building a Multilingual Named Entity-Annotated Corpus Using Annotation Projection

Spoken Language Understanding: Systems for Extracting Semantic Information from Speech

What is left to be understood in ATIS?

Spoken language understanding

Talking to machines (statistically speaking)

Inducing Multilingual Text Analysis Tools via Robust Projection across Aligned Corpora

Evaluation of Spoken Language Systems: the ATIS Domain

Ethnologue: Languages of the World, twenty-fifth edition

Exploring Cross-Lingual Transfer Learning with Unsupervised Machine Translation

New alexa features: Interactive teaching by customers

Cross-lingual Transfer Learning

Cross-lingual Name Tagging and Linking for 282 Languages

Human language technology resources for less commonly taught languages: Lessons learned toward creation of basic language resources

MeCab : Yet Another Part-of-Speech and Morphological Analyzer

Large Linguistic Areas and Language Sampling

Micro-Averaged Slot F1 (%)

Micro-averaged slot-filling F1 by language for our three models using the full dataset and the zero-shot setup

International Conference on Language Resources and Evaluation (LREC’12) , Istanbul, Turkey

Findings of the Association for Computational Lin-guistics

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL