Table Search Using a Deep Contextualized Language Model

Published in

Annual International ACM SIGIR Conference on Re...(2020)

External Links:

Generate Graph DownloadPDF

TL;DR

This paper investigates how to encode table content considering the table structure and input length limit of BERT and proposes an approach that incorporates features from prior literature on table retrieval and jointly trains them with BERT.

Abstract

Pretrained contextualized language models such as BERT have achieved impressive results on various natural language processing benchmarks. Benefiting from multiple pretraining tasks and large scale training corpora, pretrained models can capture complex syntactic word relations. In this paper, we use the deep contextualized language model BERT for the task of ad hoc table retrieval. We investigate how to encode table content considering the table structure and input length limit of BERT. We also propose an approach that incorporates features from prior literature on table retrieval and jointly trains them with BERT. In experiments on public datasets, we show that our best approach can outperform the previous state-of-the-art method and BERT baselines with a large margin under different evaluation metrics.

Authors

Zhiyu Chen

4 papers

M. Trabelsi

2 papers

Jeff Heflin

2 papers

References51 items

Leveraging Schema Labels to Enhance Dataset Search

Improved Table Retrieval Using Multiple Context Embeddings for Attributes

Stick to the Facts: Learning towards a Fidelity-oriented E-Commerce Product Description Generation

Multi-Stage Document Ranking with BERT

Universal Text Representation from BERT: An Empirical Study

Table Search Using a Deep Contextualized Language Model

Published in

Annual International ACM SIGIR Conference on Re...(2020)

External Links:

Generate Graph DownloadPDF

TL;DR

Abstract

Authors

Zhiyu Chen

4 papers

M. Trabelsi

2 papers

Jeff Heflin

2 papers

References51 items

Leveraging Schema Labels to Enhance Dataset Search

Improved Table Retrieval Using Multiple Context Embeddings for Attributes

Stick to the Facts: Learning towards a Fidelity-oriented E-Commerce Product Description Generation

Multi-Stage Document Ranking with BERT

Universal Text Representation from BERT: An Empirical Study

Brian D. Davison

3 papers

Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering

Revealing the Dark Secrets of BERT

A Study of BERT for Non-Factoid Question-Answering under Passage Length Constraints

Natural Questions: A Benchmark for Question Answering Research

RoBERTa: A Robustly Optimized BERT Pretraining Approach

Content-based table retrieval for web queries

Retrieve, Read, Rerank: Towards End-to-End Multi-Document Reading Comprehension

What Does BERT Look at? An Analysis of BERT’s Attention

Latent Retrieval for Weakly Supervised Open Domain Question Answering

Table2Vec: Neural Word and Entity Embeddings for Table Population and Retrieval

Deeper Text Understanding for IR with Contextual Neural Language Modeling

FAQ Retrieval using Query-Question Similarity and BERT-Based Query-Answer Relevance

X-BERT: eXtreme Multi-label Text Classification with using Bidirectional Encoder Representations from Transformers

Investigating the Successes and Failures of BERT for Passage Re-Ranking

Understanding the Behaviors of BERT in Ranking

CEDR: Contextualized Embeddings for Document Ranking

Simple Applications of BERT for Ad Hoc Document Retrieval

End-to-End Open-Domain Question Answering with BERTserini

Passage Re-ranking with BERT

Transformer-XL: Attentive Language Models beyond a Fixed-Length Context

Denoising Distantly Supervised Open-Domain Question Answering

Joint Training of Candidate Extraction and Answer Selection for Reading Comprehension

GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding

Ad Hoc Table Retrieval using Semantic Similarity

Neural Ranking Models with Multiple Document Fields

Challenges in Data-to-Document Generation

Attention is All you Need

Content-Based Table Retrieval for Web Queries

Enriching Word Vectors with Subword Information

Table Cell Search for Question Answering

TabEL: Entity Linking in Web Tables

A Field Relevance Model for Structured Document Retrieval

Recovering Semantics of Tables on the Web

Web-scale table census and classification

A machine learning approach for improved BM25 retrieval

Data Integration for the Relational Web

A Probabilistic Retrieval Model for Semistructured Data

WebTables: exploring the power of tables on the web

DBpedia: A Nucleus for a Web of Open Data

Simple BM25 extension to multiple weighted fields

Combining document representations for known-item search

A Machine Learning Model for Information Retrieval with Structured Documents

Effective retrieval of structured documents

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

RQ2: Could BERT with content selection methods outperform state-of-the-art performance without additional features?

RQ1: What is the performance gain of BERT with content selection methods, with respect to state-of-the-art performance?

Field of Study

Computer Science

Journal Information

Name

Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval

Venue Information

Name

Annual International ACM SIGIR Conference on Research and Development in Information Retrieval

Type

conference

URL

http://www.acm.org/sigir/

Alternate Names

International ACM SIGIR Conference on Research and Development in Information Retrieval
Int ACM SIGIR Conf Res Dev Inf Retr
SIGIR
Annu Int ACM SIGIR Conf Res Dev Inf Retr