Recurrent Neural Network for Text Classification with Multi-Task Learning

Published in

International Joint Conference on Artificial In...(2016)

External Links:

Generate Graph

TL;DR

This paper uses the multi-task learning framework to jointly learn across multiple related tasks based on recurrent neural network to propose three different mechanisms of sharing information to model text with task-specific and shared layers.

Abstract

Neural network based methods have obtained great progress on a variety of natural language processing tasks. However, in most previous works, the models are learned based on single-task supervised objectives, which often suffer from insufficient training data. In this paper, we use the multi-task learning framework to jointly learn across multiple related tasks. Based on recurrent neural network, we propose three different mechanisms of sharing information to model text with task-specific and shared layers. The entire network is trained jointly on all these tasks. Experiments on four benchmark text classification tasks show that our proposed models can improve the performance of a task with the help of other related tasks.

Authors

Pengfei Liu

10 papers

Xipeng Qiu

18 papers

Xuanjing Huang

13 papers

References39 items

Polyglot Neural Language Models: A Case Study in Cross-Lingual Phonetic Representation Learning

Exploiting Cyclic Symmetry in Convolutional Neural Networks

Many Languages, One Parser

Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism

Multi-Source Neural Translation

Recurrent Neural Network for Text Classification with Multi-Task Learning

Published in

International Joint Conference on Artificial In...(2016)

External Links:

Generate Graph

TL;DR

Abstract

Authors

Pengfei Liu

10 papers

Xipeng Qiu

18 papers

Xuanjing Huang

13 papers

References39 items

Polyglot Neural Language Models: A Case Study in Cross-Lingual Phonetic Representation Learning

Exploiting Cyclic Symmetry in Convolutional Neural Networks

Many Languages, One Parser

Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism

Multi-Source Neural Translation

Multi-Timescale Long Short-Term Memory Neural Network for Modelling Sentences and Documents

Multi-Task Learning for Multiple Language Translation

Representation Learning Using Multi-Task Deep Neural Networks for Semantic Classification and Information Retrieval

Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets

Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks

Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling

Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

Distributed Representations of Sentences and Documents

A Convolutional Neural Network for Modelling Sentences

Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank

Generating Sequences With Recurrent Neural Networks

Efficient Estimation of Word Representations in Vector Space

Semantic Compositionality through Recursive Matrix-Vector Spaces

Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions

Learning Word Vectors for Sentiment Analysis

Adaptive Subgradient Methods for Online Learning and Stochastic Optimization

Natural Language Processing (Almost) from Scratch

Word Representations: A Simple and General Method for Semi-Supervised Learning

A unified architecture for natural language processing: deep neural networks with multitask learning

Greedy Layer-Wise Training of Deep Networks

A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts

Long Short-Term Memory

Multitask Learning

Finding Structure in Time

61473092, and 61472088), the National High Technology Research and Development Program of China

Proceedings of ACL

The Journal of Machine Learning Research

Proceedings of EMNLP

Gradient Flow in Recurrent Nets: the Difficulty of Learning Long-Term Dependencies

and Jürgen Schmidhuber

What is Cognitive Science

Caruana, 1997] Rich Caruana. Multitask learning. Machine learning

Neural Computation

Field of Study

Computer Science

Journal Information

Name

ArXiv

Volume

abs/2005.00687

Venue Information

Name

International Joint Conference on Artificial Intelligence

Type

conference

URL

http://www.ijcai.org/

Alternate Names

Int Jt Conf Artif Intell
IJCAI

TL;DR

Abstract

Authors

References39 items

Polyglot Neural Language Models: A Case Study in Cross-Lingual Phonetic Representation Learning

Exploiting Cyclic Symmetry in Convolutional Neural Networks

Many Languages, One Parser

Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism

Multi-Source Neural Translation

TL;DR

Abstract

Authors

References39 items

Polyglot Neural Language Models: A Case Study in Cross-Lingual Phonetic Representation Learning

Exploiting Cyclic Symmetry in Convolutional Neural Networks

Many Languages, One Parser

Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism

Multi-Source Neural Translation

Multi-Timescale Long Short-Term Memory Neural Network for Modelling Sentences and Documents

Multi-Task Learning for Multiple Language Translation

Representation Learning Using Multi-Task Deep Neural Networks for Semantic Classification and Information Retrieval

Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets

Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks

Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling

Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

Distributed Representations of Sentences and Documents

A Convolutional Neural Network for Modelling Sentences

Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank

Generating Sequences With Recurrent Neural Networks

Efficient Estimation of Word Representations in Vector Space

Semantic Compositionality through Recursive Matrix-Vector Spaces

Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions

Learning Word Vectors for Sentiment Analysis

Adaptive Subgradient Methods for Online Learning and Stochastic Optimization

Natural Language Processing (Almost) from Scratch

Word Representations: A Simple and General Method for Semi-Supervised Learning

Et al

A unified architecture for natural language processing: deep neural networks with multitask learning

Greedy Layer-Wise Training of Deep Networks

A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts

Long Short-Term Memory

Multitask Learning

Finding Structure in Time

61473092, and 61472088), the National High Technology Research and Development Program of China

Proceedings of ACL

The Journal of Machine Learning Research

Proceedings of EMNLP

Gradient Flow in Recurrent Nets: the Difficulty of Learning Long-Term Dependencies

and Jürgen Schmidhuber

What is Cognitive Science

Caruana, 1997] Rich Caruana. Multitask learning. Machine learning

Neural Computation

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names