Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change
TL
TL;DR
A robust methodology for quantifying semantic change is developed by evaluating word embeddings against known historical changes and it is revealed that words that are more polysemous have higher rates of semantic change.
Abstract
Understanding how words change their meanings over time is key to models of language and cultural evolution, but historical data on meaning is scarce, making theories hard to develop and test. Word embeddings show promise as a diachronic tool, but have not been carefully evaluated. We develop a robust methodology for quantifying semantic change by evaluating word embeddings (PPMI, SVD, word2vec) against known historical changes. We then use this methodology to reveal statistical laws of semantic evolution. Using six historical corpora spanning four languages and two centuries, we propose two quantitative laws of semantic change: (i) the law of conformity---the rate of semantic change scales with an inverse power-law of word frequency; (ii) the law of innovation---independent of frequency, words that are more polysemous have higher rates of semantic change.
References72 items
1
Robust Statistics
2
Historical Semantic Chaining and Efficient Communication: The Case of Container Names
3
A Bayesian Model of Diachronic Meaning Change
4
Improving Distributional Similarity with Lessons Learned from Word Embeddings
5