ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs (2015-12-16T00:00:00.000000Z)

TL;DR

This work presents a general Attention Based Convolutional Neural Network (ABCNN) for modeling a pair of sentences and proposes three attention schemes that integrate mutual influence between sentences into CNNs; thus, the representation of each sentence takes into consideration its counterpart.

Abstract

How to model a pair of sentences is a critical issue in many NLP tasks such as answer selection (AS), paraphrase identification (PI) and textual entailment (TE). Most prior work (i) deals with one individual task by fine-tuning a specific system; (ii) models each sentence’s representation separately, rarely considering the impact of the other sentence; or (iii) relies fully on manually designed, task-specific linguistic features. This work presents a general Attention Based Convolutional Neural Network (ABCNN) for modeling a pair of sentences. We make three contributions. (i) The ABCNN can be applied to a wide variety of tasks that require modeling of sentence pairs. (ii) We propose three attention schemes that integrate mutual influence between sentences into CNNs; thus, the representation of each sentence takes into consideration its counterpart. These interdependent sentence pair representations are more powerful than isolated sentence representations. (iii) ABCNNs achieve state-of-the-art performance on AS, PI and TE tasks. We release code at: https://github.com/yinwenpeng/Answer_Selection.

Authors

Hinrich Schütze

22 papers

Bowen Zhou

11 papers

Bing Xiang

7 papers

TL;DR

Abstract

Authors

References59 items

Learning Transferrable Knowledge for Semantic Segmentation with Deep Convolutional Neural Network

Look and Think Twice: Capturing Top-Down Visual Attention with Feedback Convolutional Neural Networks

A Deep Architecture for Semantic Matching with Multiple Positional Sentence Representations

ABC-CNN: An Attention Based Convolutional Neural Network for Visual Question Answering

LSTM-based Deep Learning Models for non-factoid answer selection

Reasoning about Entailment with Neural Attention

WikiQA: A Challenge Dataset for Open-Domain Question Answering

A Neural Attention Model for Abstractive Sentence Summarization

Multi-Perspective Sentence Similarity Modeling with Convolutional Neural Networks

A large annotated corpus for learning natural language inference

Effective Approaches to Attention-based Neural Machine Translation

Applying deep learning to answer selection: A study and an open task

MultiGranCNN: An Architecture for General Matching of Text Chunks on Multiple Levels of Granularity

Attention-Based Models for Speech Recognition

A Hierarchical Neural Autoencoder for Paragraphs and Documents

DRAW: A Recurrent Neural Network For Image Generation

Show, Attend and Tell: Neural Image Caption Generation with Visual Attention

Multiple Object Recognition with Visual Attention

Convolutional Neural Network Architectures for Matching Natural Language Sentences

Deep Learning for Answer Sentence Selection

End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results

The application of two-level attention models in deep convolutional neural network for fine-grained image classification

Neural Machine Translation by Jointly Learning to Align and Translate

Convolutional Neural Networks for Sentence Classification

Illinois-LH: A Denotational and Distributional Approach to Semantics

ECNU: One Stone Two Birds: Ensemble of Heterogenous Measures for Semantic Relatedness and Textual Entailment

SemEval-2014 Task 1: Evaluation of Compositional Distributional Semantic Models on Full Sentences through Semantic Relatedness and Textual Entailment

UNAL-NLP: Combining Soft Cardinality Features for Semantic Textual Similarity, Relatedness and Entailment

Recurrent Models of Visual Attention

Recursive Neural Networks Can Learn Logical Semantics

Distributed Representations of Sentences and Documents

A SICK cure for the evaluation of compositional distributional semantic models

A Convolutional Neural Network for Modelling Sentences

Distributed Representations of Words and Phrases and their Compositionality

Discriminative Improvements to Distributional Sentence Similarity

Semi-Markov Phrase-Based Monolingual Alignment

Question Answering Using Enhanced Lexical Semantic Models

Automatic Coupling of Answer Extraction and Information Retrieval

A Comparison of Vector-based Representations for Semantic Composition

Re-examining Machine Translation Metrics for Paraphrase Identification

Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection

Adaptive Subgradient Methods for Online Learning and Stochastic Optimization

Discriminative Learning over Constrained Latent Representations

Tree Edit Models for Recognizing Textual Entailments, Paraphrases, and Answers to Questions

Structured retrieval for question answering

Using Semantic Roles to Improve Question Answering

What is the Jeopardy Model? A Quasi-Synchronous Grammar for QA

Cogex: A semantically and contextually enriched logic prover for question answering

Unsupervised Construction of Large Paraphrase Corpora: Exploiting Massively Parallel News Sources

ROUGE: A Package for Automatic Evaluation of Summaries

Long Short-Term Memory

WordNet: A Lexical Database for English

ATM network: goals and challenges

Signature Verification Using A "Siamese" Time Delay Neural Network

Convolutional Neural Network for Paraphrase Identification

Karl Moritz Hermann, Tomáš Kočiskỳ, and Phil Blunsom

Mapping Dependencies Trees: An Application to Question Answering

Gradient-based learning applied to document recognition

Bentz , Léon Bottou , Isabelle Guyon , Yann LeCun , Cliff Moore , Eduard Säckinger , and

Field of Study

Journal Information

Name

Page

Volume

Venue Information

Name

Type

URL

Alternate Names