RepBERT: Contextualized Text Embeddings for First-Stage Retrieval
Published in
arXiv.org(2020)External Links:
TL
TL;DR
This work proposes a different approach, called RepBERT, to represent documents and queries with fixed-length contextualized embeddings, which achieves state-of-the-art results among all initial retrieval techniques.
Abstract
Although exact term match between queries and documents is the dominant method to perform first-stage retrieval, we propose a different approach, called RepBERT, to represent documents and queries with fixed-length contextualized embeddings. The inner products of query and document embeddings are regarded as relevance scores. On MS MARCO Passage Ranking task, RepBERT achieves state-of-the-art results among all initial retrieval techniques. And its efficiency is comparable to bag-of-words methods.
References23 items
1
Dense Passage Retrieval for Open-Domain Question Answering
2
5分で分かる!? 有名論文ナナメ読み：Jacob Devlin et al. : BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding
3
REALM: Retrieval-Augmented Language Model Pre-Training
4
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
5