Dense Passage Retrieval for Open-Domain Question Answering (2020-04-10T00:00:00.000000Z)

TL;DR

This work shows that retrieval can be practically implemented using dense representations alone, where embeddings are learned from a small number of questions and passages by a simple dual-encoder framework.

Abstract

Open-domain question answering relies on efficient passage retrieval to select candidate contexts, where traditional sparse vector space models, such as TF-IDF or BM25, are the de facto method. In this work, we show that retrieval can be practically implemented using dense representations alone, where embeddings are learned from a small number of questions and passages by a simple dual-encoder framework. When evaluated on a wide range of open-domain QA datasets, our dense retriever outperforms a strong Lucene-BM25 system largely by 9%-19% absolute in terms of top-20 passage retrieval accuracy, and helps our end-to-end QA system establish new state-of-the-art on multiple open-domain QA benchmarks.

Authors

Danqi Chen

18 papers

Sewon Min

10 papers

Wen-tau Yih

9 papers

TL;DR

Abstract

Authors

References55 items

Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering

Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT

Progressively Pretrained Dense Corpus Index for Open-Domain Question Answering

REALM: Retrieval-Augmented Language Model Pre-Training

How Much Knowledge Can You Pack into the Parameters of a Language Model?

Break It Down: A Question Understanding Benchmark

Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering

Knowledge Guided Text Retrieval and Reading for Open Domain Question Answering

BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

A Discrete Hard EM Approach for Weakly Supervised Question Answering

Revealing the Importance of Semantic Retrieval for Machine Reading at Scale

Learning Dense Representations for Entity Retrieval

Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering

Natural Questions: A Benchmark for Question Answering Research

Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Index

Latent Retrieval for Weakly Supervised Open Domain Question Answering

Multi-step Retriever-Reader Interaction for Scalable Open-domain Question Answering

Poly-encoders: Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring

Real-time Inference in Multi-sentence Tasks with Deep Pretrained Transformers

Data Augmentation for BERT Fine-Tuning in Open-Domain Question Answering

End-to-End Open-Domain Question Answering with BERTserini

Passage Re-ranking with BERT

Denoising Distantly Supervised Open-Domain Question Answering

Introduction I

R3: Reinforced Ranker-Reader for Open-Domain Question Answering

Efficient Natural Language Response Suggestion for Smart Reply

TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension

Reading Wikipedia to Answer Open-Domain Questions

Billion-Scale Similarity Search with GPUs

Learning and Inference via Maximum Inner Product Search

SQuAD: 100,000+ Questions for Machine Comprehension of Text

Modeling of the Question Answering Task in the YodaQA System

Quantization based Fast Inner Product Search

Asymmetric LSH (ALSH) for Sublinear Time Maximum Inner Product Search (MIPS)

Learning deep structured semantic models for web search using clickthrough data

Semantic Parsing on Freebase from Question-Answer Pairs

Metric Learning: A Survey

Introduction to "This is Watson"

Maximum inner-product search using cone trees

Learning Discriminative Projections for Text Similarity Measures

The Probabilistic Relevance Framework: BM25 and Beyond

Latent semantic analysis

Learning to rank using gradient descent

Learning a similarity metric discriminatively, with application to face verification

An Analysis of the AskMSR Question-Answering System

Performance Issues and Error Analysis in an Open-Domain Question Answering System

Scaling question answering to the Web

Signature Verification Using A "Siamese" Time Delay Neural Network

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

born 6 November 1980) is a Norwegian stage and screen actor. He appeared in the Norwegian film ”Max Manus” and played Thor Heyerdahl in the Oscar-nominated 2012 film ”Kon-Tiki

A Profile in Statistics of Journal Articles: Fifty Years of American Documentation and the Journal of the American Society for Information Science

The TREC-8 Question Answering Track Report

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names