Towards Scalable and Reliable Capsule Networks for Challenging NLP Applications

Published in

Annual Meeting of the Association for Computati...(2019)

External Links:

Generate Graph DownloadPDF

TL;DR

An agreement score to evaluate the performance of routing processes at instance-level, an adaptive optimizer to enhance the reliability of routing, and capsule compression and partial routing to improve the scalability of capsule networks are introduced.

Abstract

Obstacles hindering the development of capsule networks for challenging NLP applications include poor scalability to large output spaces and less reliable routing processes. In this paper, we introduce: (i) an agreement score to evaluate the performance of routing processes at instance-level; (ii) an adaptive optimizer to enhance the reliability of routing; (iii) capsule compression and partial routing to improve the scalability of capsule networks. We validate our approach on two NLP tasks, namely: multi-label text classification and question answering. Experimental results show that our approach considerably improves over strong competitors on both tasks. In addition, we gain the best results in low-resource settings with few training instances.

Authors

E. Cambria

25 papers

Min Yang

4 papers

Haiyun Peng

2 papers

References57 items

COALA: A Neural Coverage-Based Approach for Long Answer Selection with Small Data

Improving the Similarity Measure of Determinantal Point Processes for Extractive Multi-Document Summarization

Text Processing Like Humans Do: Visually Attacking and Shielding NLP Systems

Towards Linear Time Neural Machine Translation with Capsule Networks

Zero-shot User Intent Detection via Capsule Neural Networks

Towards Scalable and Reliable Capsule Networks for Challenging NLP Applications

Published in

Annual Meeting of the Association for Computati...(2019)

External Links:

Generate Graph DownloadPDF

TL;DR

Abstract

Authors

E. Cambria

25 papers

Min Yang

4 papers

Haiyun Peng

2 papers

References57 items

COALA: A Neural Coverage-Based Approach for Long Answer Selection with Small Data

Improving the Similarity Measure of Determinantal Point Processes for Extractive Multi-Document Summarization

Text Processing Like Humans Do: Visually Attacking and Shielding NLP Systems

Towards Linear Time Neural Machine Translation with Capsule Networks

Zero-shot User Intent Detection via Capsule Neural Networks

Steffen Eger

3 papers

Knowledge as A Bridge: Improving Cross-domain Answer Selection with External Knowledge

Knowledge-aware Attentive Neural Network for Ranking Question Answer Pairs

Fast dynamic routing based on weighted kernel density estimation

A Multi-View Fusion Neural Network for Answer Selection

Investigating Capsule Networks with Dynamic Routing for Text Classification

Achieving Human Parity on Automatic Chinese to English News Translation

Matrix capsules with EM routing

Dynamic Routing Between Capsules

Deep Learning for Extreme Multi-label Text Classification

Learning to Rank Question Answer Pairs with Holographic Dual LSTM Architecture

EviNets: Neural Networks for Combining Evidence Signals for Factoid Question Answering

An End-to-End Model for Question Answering over Knowledge Base with Cross-Attention Combining Global Knowledge

Improved Representation Learning for Question Answer Matching

Bag of Tricks for Efficient Text Classification

PD-Sparse : A Primal and Dual Sparse Approach to Extreme Multiclass and Multilabel Classification

Simple Question Answering by Attentive Convolutional Neural Network

Attentive Pooling Networks

Human-level concept learning through probabilistic program induction

Sparse Local Embeddings for Extreme Multi-label Classification

A Deep Architecture for Semantic Matching with Multiple Positional Sentence Representations

Learning to Rank Short Text Pairs with Convolutional Deep Neural Networks

Semantic Parsing via Staged Query Graph Generation: Question Answering with Knowledge Base

Convolutional Neural Tensor Network Architecture for Community-Based Question Answering

A Long Short-Term Memory Model for Answer Sentence Selection in Question Answering

Large-scale Simple Question Answering with Memory Networks

Deep Learning for Answer Sentence Selection

Effective Use of Word Order for Text Categorization with Convolutional Neural Networks

GloVe: Global Vectors for Word Representation

Convolutional Neural Networks for Sentence Classification

FastXML: a fast, accurate and stable tree-classifier for extreme multi-label learning

Information Extraction over Structured Data: Question Answering with Freebase

Robust Bloom Filters for Large MultiLabel Classification Tasks

Large-scale Multi-label Learning with Missing Labels

Efficient Multi-label Classification with Many Labels

Label Partitioning For Sublinear Ranking

Multi-label learning with millions of labels: recommending advertiser bid phrases for web pages

Multilabel Classification using Bayesian Compressed Sensing

Feature-aware Label Space Dimension Reduction for Multi-label Classification

The Landmark Selection Method for Multiple Output Prediction

Multi-label Classification with Error-correcting Codes

Transforming Auto-Encoders

Multi-Label Prediction via Compressed Sensing

Efficient Pairwise Multilabel Classification for Large-Scale Problems in the Legal Domain

Extracting shared subspace for multi-label classification

What is the Jeopardy Model? A Quasi-Synchronous Grammar for QA

RCV1: A New Benchmark Collection for Text Categorization Research

Mean Shift: A Robust Approach Toward Feature Space Analysis

MCapsNet: Capsule Network for Text with Multi-Task Learning

Attention-Based Capsule Network with Dynamic Routing for Relation Extraction

Extracting Shared Subspaces for Multi-label Classiﬁcation

Open-Domain Question Answering from Large Text Collections

Field of Study

Computer Science

Journal Information

Name

ArXiv

Volume

abs/2005.00687

Venue Information

Name

Annual Meeting of the Association for Computational Linguistics

Type

conference

URL

https://www.aclweb.org/anthology/venues/acl/

Alternate Names

Annu Meet Assoc Comput Linguistics
Meeting of the Association for Computational Linguistics
ACL
Meet Assoc Comput Linguistics

TL;DR

Abstract

Authors

References57 items

COALA: A Neural Coverage-Based Approach for Long Answer Selection with Small Data

Improving the Similarity Measure of Determinantal Point Processes for Extractive Multi-Document Summarization

Text Processing Like Humans Do: Visually Attacking and Shielding NLP Systems

Towards Linear Time Neural Machine Translation with Capsule Networks

Zero-shot User Intent Detection via Capsule Neural Networks

TL;DR

Abstract

Authors

References57 items

COALA: A Neural Coverage-Based Approach for Long Answer Selection with Small Data

Improving the Similarity Measure of Determinantal Point Processes for Extractive Multi-Document Summarization

Text Processing Like Humans Do: Visually Attacking and Shielding NLP Systems

Towards Linear Time Neural Machine Translation with Capsule Networks

Zero-shot User Intent Detection via Capsule Neural Networks

Knowledge as A Bridge: Improving Cross-domain Answer Selection with External Knowledge

Knowledge-aware Attentive Neural Network for Ranking Question Answer Pairs

Fast dynamic routing based on weighted kernel density estimation

A Multi-View Fusion Neural Network for Answer Selection

Investigating Capsule Networks with Dynamic Routing for Text Classification

Achieving Human Parity on Automatic Chinese to English News Translation

Matrix capsules with EM routing

Dynamic Routing Between Capsules

Deep Learning for Extreme Multi-label Text Classification

Learning to Rank Question Answer Pairs with Holographic Dual LSTM Architecture

EviNets: Neural Networks for Combining Evidence Signals for Factoid Question Answering

An End-to-End Model for Question Answering over Knowledge Base with Cross-Attention Combining Global Knowledge

Improved Representation Learning for Question Answer Matching

Bag of Tricks for Efficient Text Classification

PD-Sparse : A Primal and Dual Sparse Approach to Extreme Multiclass and Multilabel Classification

Simple Question Answering by Attentive Convolutional Neural Network

Attentive Pooling Networks

Human-level concept learning through probabilistic program induction

Sparse Local Embeddings for Extreme Multi-label Classification

A Deep Architecture for Semantic Matching with Multiple Positional Sentence Representations

Learning to Rank Short Text Pairs with Convolutional Deep Neural Networks

Semantic Parsing via Staged Query Graph Generation: Question Answering with Knowledge Base

Convolutional Neural Tensor Network Architecture for Community-Based Question Answering

A Long Short-Term Memory Model for Answer Sentence Selection in Question Answering

Large-scale Simple Question Answering with Memory Networks

Deep Learning for Answer Sentence Selection

Effective Use of Word Order for Text Categorization with Convolutional Neural Networks

GloVe: Global Vectors for Word Representation

Convolutional Neural Networks for Sentence Classification

FastXML: a fast, accurate and stable tree-classifier for extreme multi-label learning

Information Extraction over Structured Data: Question Answering with Freebase

Robust Bloom Filters for Large MultiLabel Classification Tasks

Large-scale Multi-label Learning with Missing Labels

Efficient Multi-label Classification with Many Labels

Label Partitioning For Sublinear Ranking

Multi-label learning with millions of labels: recommending advertiser bid phrases for web pages

Multilabel Classification using Bayesian Compressed Sensing

Feature-aware Label Space Dimension Reduction for Multi-label Classification

The Landmark Selection Method for Multiple Output Prediction

Multi-label Classification with Error-correcting Codes

Transforming Auto-Encoders

Multi-Label Prediction via Compressed Sensing

Efficient Pairwise Multilabel Classification for Large-Scale Problems in the Legal Domain

Extracting shared subspace for multi-label classification

What is the Jeopardy Model? A Quasi-Synchronous Grammar for QA

RCV1: A New Benchmark Collection for Text Categorization Research

Mean Shift: A Robust Approach Toward Feature Space Analysis

MCapsNet: Capsule Network for Text with Multi-Task Learning

Attention-Based Capsule Network with Dynamic Routing for Relation Extraction

Extracting Shared Subspaces for Multi-label Classiﬁcation

Open-Domain Question Answering from Large Text Collections

Kernel Smoothing

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names