NTUA-SLP at SemEval-2018 Task 1: Predicting Affective Content in Tweets with Deep Attentive RNNs and Transfer Learning (2018-04-18T00:00:00.000000Z)

TL;DR

This paper proposes a Bi-LSTM architecture equipped with a multi-layer self attention mechanism that improves the model performance and allows us to identify salient words in tweets, as well as gain insight into the models making them more interpretable.

Abstract

In this paper we present deep-learning models that submitted to the SemEval-2018 Task 1 competition: “Affect in Tweets”. We participated in all subtasks for English tweets. We propose a Bi-LSTM architecture equipped with a multi-layer self attention mechanism. The attention mechanism improves the model performance and allows us to identify salient words in tweets, as well as gain insight into the models making them more interpretable. Our model utilizes a set of word2vec word embeddings trained on a large collection of 550 million Twitter messages, augmented by a set of word affective features. Due to the limited amount of task-specific training data, we opted for a transfer learning approach by pretraining the Bi-LSTMs on the dataset of Semeval 2017, Task 4A. The proposed approach ranked 1st in Subtask E “Multi-Label Emotion Classification”, 2nd in Subtask A “Emotion Intensity Regression” and achieved competitive results in other subtasks.

Authors

Shrikanth S. Narayanan

2 papers

A. Potamianos

4 papers

Alexandra Chronopoulou

2 papers

TL;DR

Abstract

Authors

References55 items

SemEval-2018 Task 1: Affect in Tweets

Deep Contextualized Word Representations

Fine-tuned Language Models for Text Classification

Universal Language Model Fine-tuning for Text Classification

Automatic differentiation in PyTorch

DataStories at SemEval-2017 Task 4: Deep LSTM with Attention for Message-level and Topic-based Sentiment Analysis

SemEval-2017 Task 4: Sentiment Analysis in Twitter

Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm

Deep Learning for User Comment Moderation

A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks

Charagram: Embedding Words and Sentences via Character n-grams

SwissCheese at SemEval-2016 Task 4: Sentiment Classification Using an Ensemble of Convolutional Neural Networks with Distant Supervision

VQA: Visual Question Answering

Adam: A Method for Stochastic Optimization

Fully convolutional networks for semantic segmentation

Neural Machine Translation by Jointly Learning to Align and Translate

Political Tendency Identification in Twitter using Sentiment Analysis Techniques

Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

DeepFace: Closing the Gap to Human-Level Performance in Face Verification

Sentiment Analysis of Short Informal Texts

CNN Features Off-the-Shelf: An Astounding Baseline for Recognition

Distributional Semantic Models for Affective Text Analysis

Distributed Representations of Words and Phrases and their Compositionality

NRC-Canada: Building the State-of-the-Art in Sentiment Analysis of Tweets

CROWDSOURCING A WORD–EMOTION ASSOCIATION LEXICON

Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures

On the difficulty of training recurrent neural networks

Combining lexicon and learning based approaches for concept-level sentiment analysis

A New ANEW: Evaluation of a Word List for Sentiment Analysis in Microblogs

Scikit-learn: Machine Learning in Python

Twitter mood predicts the stock market

Emotions Evoked by Common Words and Phrases: Using Mechanical Turk to Create an Emotion Lexicon

Software Framework for Topic Modelling with Large Corpora

Predicting Elections with Twitter: What 140 Characters Reveal about Political Sentiment

Modeling Public Mood and Emotion: Twitter Sentiment and Socio-Economic Phenomena

Beautiful Data: The Stories Behind Elegant Data Solutions

ImageNet: A large-scale hierarchical image database

A unified architecture for natural language processing: deep neural networks with multitask learning

Characterization of the Affective Norms for English Words by discrete emotional categories

Extensions of the Paivio, Yuille, and Madigan (1968) norms

MRC Psycholinguistic Database

Long Short-Term Memory

Learning long-term dependencies with gradient descent is difficult

Speech and Language Processing

Character and Subword-Based Word Representation for Neural Language Modeling Prediction

Emotion Detection from Text: Survey

SENSEI-LIF at SemEval-2016 Task 4: Polarity embedding fusion for robust sentiment analysis

Valence, arousal and dominance estimation for English, German, Greek, Portuguese and Spanish lexica using semantic models

Dropout: a simple way to prevent neural networks from overfitting

Exploiting Topic based Twitter Sentiment for Stock Prediction

Discovering Consumer Insight from Twitter via Sentiment Analysis

Twitter Sentiment Classiﬁcation using Distant Supervision

Gradient Flow in Recurrent Nets: the Difficulty of Learning Long-Term Dependencies

Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition

Affective Norms for English Words (ANEW): Instruction Manual and Affective Ratings

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names