Learning to Learn to Disambiguate: Meta-Learning for Few-Shot Word Sense Disambiguation (2020-04-29T00:00:00.000000Z)

TL;DR

This paper proposes a meta-learning framework for few-shot word sense disambiguation (WSD), where the goal is to learn todisambiguate unseen words from only a few labeled instances.

Abstract

The success of deep learning methods hinges on the availability of large training datasets annotated for the task of interest. In contrast to human intelligence, these methods lack versatility and struggle to learn and adapt quickly to new tasks, where labeled data is scarce. Meta-learning aims to solve this problem by training a model on a large number of few-shot tasks, with an objective to learn new tasks quickly from a small number of examples. In this paper, we propose a meta-learning framework for few-shot word sense disambiguation (WSD), where the goal is to learn to disambiguate unseen words from only a few labeled instances. Meta-learning approaches have so far been typically tested in an N-way, K-shot classification setting where each task has N classes with K examples per class. Owing to its nature, WSD deviates from this controlled setup and requires the models to handle a large number of highly unbalanced classes. We extend several popular meta-learning approaches to this scenario, and analyze their strengths and weaknesses in this new challenging setting.

Authors

H. Yannakoudakis

7 papers

Ekaterina Shutova

6 papers

Pushkar Mishra

5 papers

TL;DR

Abstract

Authors

References76 items

Breaking Through the 80% Glass Ceiling: Raising the State of the Art in Word Sense Disambiguation by Incorporating Knowledge Graph Information

Tracking by Instance Detection: A Meta-Learning Approach

Meta-learning curiosity algorithms

Meta-Learning without Memorization

MetalGAN: Multi-Domain Label-Less Image Synthesis Using cGANs and Meta-Learning

PyTorch: An Imperative Style, High-Performance Deep Learning Library

Learning to Few-Shot Learn Across Diverse Natural Language Classification Tasks

Hierarchical Attention Prototypical Networks for Few-Shot Text Classification

Improved Word Sense Disambiguation Using Pre-Trained Contextualized Word Representations

Generalized Inner Loop Meta-Learning

Meta-Learning Initializations for Image Segmentation

Learning to Learn and Predict: A Meta-Learning Approach for Multi-Label Classification

Investigating Meta-Learning Algorithms for Low-Resource Natural Language Understanding Tasks

GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledge

Meta-Learning Improves Lifelong Relation Extraction

Model-Agnostic Meta-Learning for Relation Classification with Limited Supervision

Few-Shot Representation Learning for Out-Of-Vocabulary Words

Zero-shot Word Sense Disambiguation using Sense Definition Embeddings

Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP

Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples

Induction Networks for Few-Shot Text Classification

Learning and Evaluating General Linguistic Intelligence

Attentive Task-Agnostic Meta-Learning for Few-Shot Text Classification

Meta-Learning for Low-Resource Neural Machine Translation

Diverse Few-Shot Text Classification with Multiple Metrics

GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding

The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks

On First-Order Meta-Learning Algorithms

Reptile: a Scalable Metalearning Algorithm

Deep Contextualized Word Representations

Learning to Compare: Relation Network for Few-Shot Learning

Neural Sequence Learning Models for Word Sense Disambiguation

Attention is All you Need

Word Sense Disambiguation: A Unified Evaluation Framework and Empirical Comparison

Prototypical Networks for Few-shot Learning

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks

Meta Networks

Learning to reinforcement learn

Optimization as a Model for Few-Shot Learning

RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning

context2vec: Learning Generic Context Embedding with Bidirectional LSTM

Embeddings for Word Sense Disambiguation: An Evaluation Study

Meta-Learning with Memory-Augmented Neural Networks

Matching Networks for One Shot Learning

Word Sense Disambiguation using a Bidirectional LSTM

Semi-supervised Word Sense Disambiguation with Neural Models

Human-level concept learning through probabilistic program induction

A large annotated corpus for learning natural language inference

AutoExtend: Extending Word Embeddings to Embeddings for Synsets and Lexemes

Adam: A Method for Stochastic Optimization

GloVe: Global Vectors for Word Representation

Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

Entity Linking meets Word Sense Disambiguation: a Unified Approach

Random Walks for Knowledge-Based Word Sense Disambiguation

Word Sense Disambiguation Improves Information Retrieval

It Makes Sense: A Wide-Coverage Word Sense Disambiguation System for Free Text

Word sense disambiguation: A survey

Word Sense Disambiguation Improves Statistical Machine Translation

An Empirical Evaluation of Knowledge Sources and Learning Algorithms for Word Sense Disambiguation

Long Short-Term Memory

Using a Semantic Concordance for Sense Identification

Multitask Learning: A Knowledge-Based Source of Inductive Bias

Learning a synaptic learning rule

Introduction to WordNet: An On-line Lexical Database

Automatic sense disambiguation using machine readable dictionaries: how to tell a pine cone from an ice cream cone

Meta Relational Learning for Few-Shot Link Prediction in Knowledge Graphs

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Learning how to learn

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

Siamese Neural Networks for One-Shot Image Recognition

Semi-Supervised Word Sense Disambiguation Using Word Embeddings in General and Specific Domains

Visualizing Data using t-SNE

Evolutionary principles in self-referential learning, or on learning how to learn: The meta-meta-. hook

for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)

9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)

The meta-meta-meta...-hook. Diploma thesis