Skip-gram word embeddings in hyperbolic space
Published in
arXiv.org(2018)External Links:
TL
TL;DR
The results demonstrate the potential of hyperbolic word embeddings, particularly in low dimensions, though without clear superiority over their Euclidean counterparts, and discuss problems in the formulation of the analogy task resulting from the curvature ofhyperbolic space.
Abstract
Recent work has demonstrated that embeddings of tree-like graphs in hyperbolic space surpass their Euclidean counterparts in performance by a large margin. Inspired by these results and scale-free structure in the word co-occurrence graph, we present an algorithm for learning word embeddings in hyperbolic space from free text. An objective function based on the hyperbolic distance is derived and included in the skip-gram negative-sampling architecture of word2vec. The hyperbolic word embeddings are then evaluated on word similarity and analogy benchmarks. The results demonstrate the potential of hyperbolic word embeddings, particularly in low dimensions, though without clear superiority over their Euclidean counterparts. We further discuss subtleties in the formulation of the analogy task in curved spaces.
References32 items
1
Differential Geometry
2
Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometry
3
Large-Margin Classification in Hyperbolic Space
4
Embedding Text in Hyperbolic Spaces
5