natural-language-processing-9

Bias Detection

3260 papers • 126 benchmarks • 313 datasets

Bias detection is the task of detecting and measuring racism, sexism and otherwise discriminatory behavior in a model (Source: https://stereoset.mit.edu/)

(Image credit: Papersgraph)

Benchmarks

These leaderboards are used to track progress in bias-detection-9

Trend

Dataset

Best Model

Actions

StereoSet

Wiki Neutrality Corpus

PlantVillage_8px

Libraries

i

Use these libraries to find bias-detection-9 models and implementations

Datasets

Subtasks

Selection bias

Most implemented papers

Automated Dependence Plots

Liu Leqi, David I. Inouye, Joon Sik Kim, Bryon Aragam, Pradeep Ravikumar•Sun Dec 01 2019

The usefulness of the automated dependence plots (ADP) across multiple use-cases and datasets including model selection, bias detection, understanding out-of-sample behavior, and exploring the latent space of a generative model is demonstrated.

2

Content

PlantVillage_8px

ICAT LLM bias

0

Paper Graph

StereoSet: Measuring stereotypical bias in pretrained language models

Siva Reddy, Moin Nadeem, Anna Bethke•Sun Apr 19 2020

StereoSet, a large-scale natural English dataset to measure stereotypical biases in four domains: gender, profession, race, and religion, is presented and it is shown that popular models like BERT, GPT-2, RoBERTa, and XLnet exhibit strong stereotypical biases.

1240 0

Paper Graph

Bipol: Multi-Axes Evaluation of Bias with Explainability in Benchmark Datasets

Tosin P. Adewumi, M. Liwicki, Isabella Sodergren, Lama Alkhaled, Sana Sabry, F. Liwicki•Fri Jan 27 2023

B bipol, a novel multi-axes bias metric with explainability, is used to estimate and explain how much bias exists in five English NLP benchmark datasets and two Swedish datasets for bias, along multiple axes.

4 0

Paper Graph

Measuring Gender Bias in Word Embeddings across Domains and Discovering New Gender Bias Word Categories

K. Chaloner, Alfredo Maldonado•Mon Dec 31 2018

This paper applies the WEAT bias detection method to four sets of word embeddings trained on corpora from four different domains: news, social networking, biomedical and a gender-balanced corpus extracted from Wikipedia, and finds that some domains are definitely more prone to gender bias than others.

54 0

Paper Graph

Multilingual sentence-level bias detection in Wikipedia

D. Aleksandrova, François Lareau, P. Ménard•Mon Oct 21 2019

A multilingual method for the extraction of biased sentences from Wikipedia is proposed, and it is used to create corpora in Bulgarian, French and English and exploit the data with well-known classification methods to detect biased sentences.

20 0

Paper Graph

Predicting the Leading Political Ideology of YouTube Channels Using Acoustic, Textual, and Metadata Information

Preslav Nakov, Ivan Koychev, Yoan Dinkov, Ahmed Ali•Sat Sep 14 2019

The analysis shows that the use of acoustic signal helped to improve bias detection by more than 6% absolute over using text and metadata only, and the dataset is released to the research community, hoping to help advance the field of multi-modal political bias detection.

32 0

Paper Graph

Towards Detection of Subjective Bias using Contextualized Word Embeddings

Kartikey Pant, R. Mamidi, Tanvi Dadu•Sat Feb 15 2020

This work performs comprehensive experiments for detecting subjective bias using BERT-based models on the Wiki Neutrality Corpus (WNC), and proposes BERT -based ensembles that outperform state-of-the-art methods like BERTlarge by a margin of 5.6 F1 score.

25 0

Paper Graph

Adding a benchmark result helps the community track progress.