natural-language-processing-9

Hateful Meme Classification

3260 papers • 126 benchmarks • 313 datasets

Hateful meme classification aims to detect harmful content within the text or images of memes.

(Image credit: Papersgraph)

Benchmarks

These leaderboards are used to track progress in hateful-meme-classification-18

Trend

Dataset

Best Model

Actions

HarMeme

PrideMM

Harm-P

Libraries

i

Use these libraries to find hateful-meme-classification-18 models and implementations

Datasets

Subtasks

No subtasks available.

Most implemented papers

Learning Transferable Visual Models From Natural Language Supervision

I. Sutskever, Alec Radford, Jong Wook Kim, Chris Hallacy, A. Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger•Thu Feb 25 2021

It is demonstrated that the simple pre-training task of predicting which caption goes with which image is an efficient and scalable way to learn SOTA image representations from scratch on a dataset of 400 million (image, text) pairs collected from the internet.

39743

Content

Hateful Memes

0

Paper Graph

Hate-CLIPper: Multimodal Hateful Meme Classification based on Cross-modal Interaction of CLIP Features

Gokul Karthik Kumar, K. Nandakumar•Tue Oct 11 2022

The Hate-CLIPper architecture is proposed, which explicitly models the cross-modal interactions between the image and text representations obtained using Contrastive Language-Image Pre-training (CLIP) encoders via a feature interaction matrix (FIM).

90 0

Paper Graph

Decoding the Underlying Meaning of Multimodal Hateful Memes

Roy Ka-Wei Lee, Ming Shan Hee, Wen-Haw Chong•Sat May 27 2023

Hateful meme with Reasons Dataset (HatReD) is introduced, which is a new multimodal hateful meme dataset annotated with the underlying hateful contextual reasons and a new conditional generation task that aims to automatically generate underlying reasons to explain hateful memes.

61 0

Paper Graph

Mapping Memes to Words for Multimodal Hateful Meme Classification

A. Bimbo, Alberto Baldrati, Lorenzo Agnolucci, Marco Bertini, Giovanni Burbi•Sun Oct 01 2023

A novel approach named ISSUES is proposed for multimodal hateful meme classification that leverages a pre-trained CLIP vision-language model and the textual inversion technique to effectively capture the multimodal semantic content of the memes.

26 0

Paper Graph

Improving Hateful Meme Detection through Retrieval-Guided Contrastive Learning

Weizhe Lin, Jingbiao Mei, Jinghong Chen, Bill Byrne, Marcus Tomalin•Mon Nov 13 2023

This work demonstrates a retrieval-based hateful memes detection system, which is capable of identifying hatefulness based on data unseen in training, and allows developers to update the hateful memes detection system by simply adding new examples without retraining.

24 0

Paper Graph

Towards Explainable Harmful Meme Detection through Multimodal Debate between Large Language Models

Ziyang Luo, Jing Ma, Hongzhan Lin, Wei Gao, Bo Wang, Ruichao Yang•Tue Jan 23 2024

This paper proposes an explainable approach to detect harmful memes, inspired by the powerful capacity of Large Language Models on text generation and reasoning, and proposes to fine-tune a small language model as the debate judge for harmfulness inference.

35 0

Paper Graph

MemeCLIP: Leveraging CLIP Representations for Multimodal Meme Classification

Siddhant Bikram Shah, Shuvam Shiwakoti, Maheep Chaudhary, Haohan Wang•Sun Sep 22 2024

This study introduces a novel dataset PrideMM comprising 5,063 text-embedded images associated with the LGBTQ+ Pride movement, thereby addressing a serious gap in existing resources and proposes a novel framework MemeCLIP for efficient downstream learning while preserving the knowledge of the pre-trained CLIP model.

28 0

Paper Graph

Prompt-enhanced Network for Hateful Meme Classification

Junxi Liu, Yanyan Feng, Jiehai Chen, Yun Xue, Fenghuan Li•Wed Jul 31 2024

Pen—a prompt-enhanced network framework based on the prompt learning approach, which surpasses manual prompt methods, showcasing superior generalization and classification accuracy in hateful meme classification tasks.

3 0

Paper Graph

Robust Adaptation of Large Multimodal Models for Retrieval Augmented Hateful Meme Detection

Weizhe Lin, Jingbiao Mei, Bill Byrne, Jinghong Chen, Guangyu Yang•Mon Feb 17 2025

This work proposes a robust adaptation framework for hateful meme detection that enhances in-domain accuracy and cross-domain generalization while preserving the general vision-language capabilities of LMMs and achieves improved robustness under adversarial attacks compared to SFT models.

6 0

Paper Graph

Adding a benchmark result helps the community track progress.