DuluthNLP at SemEval-2021 Task 7: Fine-Tuning RoBERTa Model for Humor Detection and Offense Rating (2021-01-01T00:00:00.000000Z)

TL;DR

It is shown that optimizing hyper-parameters for learning rate, batch size and number of epochs can increase the accuracy and F1 score for humor detection.

Abstract

This paper presents the DuluthNLP submission to Task 7 of the SemEval 2021 competition on Detecting and Rating Humor and Offense. In it, we explain the approach used to train the model together with the process of fine-tuning our model in getting the results. We focus on humor detection, rating, and of-fense rating, representing three out of the four subtasks that were provided. We show that optimizing hyper-parameters for learning rate, batch size and number of epochs can increase the accuracy and F1 score for humor detection

Authors

Samuel Akrah

1 papers

DuluthNLP at SemEval-2021 Task 7: Fine-Tuning RoBERTa Model for Humor Detection and Offense Rating

TL;DR

Abstract

Authors

References26 items

Transformers: State-of-the-Art Natural Language Processing

SemEval-2020 Task 7: Assessing Humor in Edited News Headlines

SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020)

UHH-LT at SemEval-2020 Task 12: Fine-Tuning of Pre-Trained Transformer Networks for Offensive Language Detection

Humor Detection: A Transformer Gets the Last Laugh

RoBERTa: A Robustly Optimized BERT Pretraining Approach

Energy and Policy Considerations for Deep Learning in NLP

Amobee at SemEval-2019 Tasks 5 and 6: Multiple Choice CNN Over Contextual Embedding

SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)

Transformer-XL: Attentive Language Models beyond a Fixed-Length Context

Robust Neural Machine Translation with Joint Textual and Phonetic Embedding

SemEval-2018 Task 3: Irony Detection in English Tweets

SemEval-2017 Task 6: #HashtagWars: Learning a Sense of Humor

Attention is All you Need

Bayesian Optimization for Machine Learning : A Practical Guidebook

An evaluation of sequential model-based optimization for expensive blackbox functions

Baselines and Bigrams: Simple, Good Sentiment and Topic Classification

Random Search for Hyper-Parameter Optimization

Sequential Model-Based Optimization for General Algorithm Configuration

SemEval 2021 Task 7: HaHackathon, Detecting and Rating Humor and Offense

Hitachi at SemEval-2020 Task 8: Simple but Effective Modality Ensemble for Meme Emotion Recognition

Experiment tracking with weights and biases. Software available from wandb

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Simple transformers

Algorithm 1, adopted from SigOpt 8

better than manual selection, it is computationally expensive

Field of Study

Venue Information

Name

Type

URL

Alternate Names