BilBOWA: Fast Bilingual Distributed Representations without Word Alignments
Published in
International Conference on Machine Learning(2014)External Links:
TL
TL;DR
It is shown that bilingual embeddings learned using the proposed BilBOWA model outperform state-of-the-art methods on a cross-lingual document classification task as well as a lexical translation task on WMT11 data.
Abstract
We introduce BilBOWA (Bilingual Bag-of-Words without Alignments), a simple and computationally-efficient model for learning bilingual distributed representations of words which can scale to large monolingual datasets and does not require word-aligned parallel training data. Instead it trains directly on monolingual data and extracts a bilingual signal from a smaller set of raw-text sentence-aligned data. This is achieved using a novel sampled bag-of-words cross-lingual objective, which is used to regularize two noise-contrastive language models for efficient cross-lingual feature learning. We show that bilingual embeddings learned using the proposed model outperform state-of-the-art methods on a cross-lingual document classification task as well as a lexical translation task on WMT11 data.
References25 items
1
GloVe: Global Vectors for Word Representation
2
Improving Vector Space Word Representations Using Multilingual Correlation
3
word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method
4
An Autoencoder Approach to Learning Bilingual Word Representations
5