A Simple Approach to Learn Polysemous Word Embeddings (2017-07-06T00:00:00.000000Z)

TL;DR

This paper proposes a simple method to learn a word representation, given any context, that only requires learning the usual single sense representation, and coefficients that can be learnt via a single pass over the data.

Abstract

Many NLP applications require disambiguating polysemous words. Existing methods that learn polysemous word vector representations involve first detecting various senses and optimizing the sense-specific embeddings separately, which are invariably more involved than single sense learning methods such as word2vec. Evaluating these methods is also problematic, as rigorous quantitative evaluations in this space is limited, especially when compared with single-sense embeddings. In this paper, we propose a simple method to learn a word representation, given any context. Our method only requires learning the usual single sense representation, and coefficients that can be learnt via a single pass over the data. We propose several new test sets for evaluating word sense induction, relevance detection, and contextual word similarity, significantly supplementing the currently available tests. Results on these and other tests show that while our method is embarrassingly simple, it achieves excellent results when compared to the state of the art models for unsupervised polysemous word representation learning.

Authors

Yifan Sun

3 papers

Nikhil S. Rao

2 papers

Weicong Ding

1 papers

A Simple Approach to Learn Polysemous Word Embeddings

TL;DR

Abstract

Authors

References25 items

Beyond Bilingual: Multi-sense Word Embeddings using Multilingual Context

Word Sense Disambiguation: A Unified Evaluation Framework and Empirical Comparison

Bilingual Learning of Multi-sense Embeddings with Discrete Autoencoders

Linear Algebraic Structure of Word Senses, with Applications to Polysemy

Learning Context-Sensitive Word Embeddings with Neural Tensor Skip-Gram Model

Skip-Thought Vectors

Do Multi-Sense Embeddings Improve Natural Language Understanding?

Improving Distributional Similarity with Lessons Learned from Word Embeddings

Breaking Sticks and Ambiguities with Adaptive Skip-gram

Topical Word Embeddings

Neural Word Embedding as Implicit Matrix Factorization

A Unified Model for Word Sense Representation and Disambiguation

Efficient Non-parametric Estimation of Multiple Embeddings per Word in Vector Space

GloVe: Global Vectors for Word Representation

Learning Sense-specific Word Embeddings By Exploiting Bilingual Resources

A Probabilistic Model for Learning Multi-Prototype Word Embeddings

Distributed Representations of Sentences and Documents

Distributed Representations of Words and Phrases and their Compositionality

Efficient Estimation of Word Representations in Vector Space

Improving Word Representations via Global Context and Multiple Word Prototypes

A Mixture Model with Sharing for Lexical Semantics

Multi-Prototype Vector-Space Models of Word Meaning

Domain-Specific Sense Distributions and Predominant Sense Acquisition

Eigenwords: spectral word embeddings

Combining Local Context and Wordnet Similarity for Word Sense Identification

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names