Analysing Lexical Semantic Change with Contextualised Word Representations
TL
TL;DR
This paper presents the first unsupervised approach to lexical semantic change that makes use of contextualised word representations, a novel method that exploits the BERT neural language model to obtain representations of word usages, clusters these representations into usage types, and measures change along time with three proposed metrics.
Abstract
This paper presents the first unsupervised approach to lexical semantic change that makes use of contextualised word representations. We propose a novel method that exploits the BERT neural language model to obtain representations of word usages, clusters these representations into usage types, and measures change along time with three proposed metrics. We create a new evaluation dataset and show that the model representations and the detected semantic shifts are positively correlated with human judgements. Our extensive qualitative analysis demonstrates that our method captures a variety of synchronic and diachronic linguistic phenomena. We expect our work to inspire further research in this direction.
References61 items
1
UiO-UvA at SemEval-2020 Task 1: Contextualised Embeddings for Lexical Semantic Change Detection
2
Capturing Evolution in Word Usage: Just Add More Clusters?
3
Does BERT Make Any Sense? Interpretable Word Sense Disambiguation with Contextualized Embeddings
4
Diachronic Sense Modeling with Deep Contextualized Word Embeddings: An Ecological View
5