Underspecification in Language Modeling Tasks: A Causality-Informed Study of Gendered Pronoun Resolution (2022-09-30T00:00:00.000000Z)

TL;DR

A simple causal mechanism is introduced to describe the role underspecification plays in the generation of spurious correlations and directly informs the development of two lightweight black-box evaluation methods.

Abstract

Modern language modeling tasks are often underspecified: for a given token prediction, many words may satisfy the user’s intent of producing natural language at inference time, however only one word will minimize the task’s loss function at training time. We introduce a simple causal mechanism to describe the role underspecification plays in the generation of spurious correlations. Despite its simplicity, our causal model directly informs the development of two lightweight black-box evaluation methods, that we apply to gendered pronoun resolution tasks on a wide range of LLMs to 1) aid in the detection of inference-time task underspecification by exploiting 2) previously unreported gender vs. time and gender vs. location spurious correlations on LLMs with a range of A) sizes: from BERT-base to GPT-3.5, B) pre-training objectives: from masked & autoregressive language modeling to a mixture of these objectives, and C) training stages: from pre-training only to reinforcement learning from human feedback (RLHF). Code and open-source demos available at https://github.com/2dot71mily/uspec.

Authors

Emily McMilin

1 papers

TL;DR

Abstract

Authors

References55 items

Causal Parrots: Large Language Models May Talk Causality But Are Not Causal

A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Models

Causally Testing Gender Bias in LLMs: A Case Study on Occupational Bias

Are All Spurious Features in Natural Language Alike? An Analysis through a Causal Lens

A Causal Framework to Quantify the Robustness of Mathematical Reasoning with Language Models

Scaling Instruction-Finetuned Language Models

Measuring Causal Effects of Data Statistics on Language Model's 'Factual' Predictions

UL2: Unifying Language Learning Paradigms

Training Compute-Optimal Large Language Models

Training language models to follow instructions with human feedback

Diversify and Disambiguate: Learning From Underspecified Data

Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Tests

Encoding Weights of Irregular Sparsity for Fixed-to-Fixed Model Compression

Underspecification Presents Challenges for Credibility in Modern Machine Learning

Measuring and Reducing Gendered Correlations in Pre-trained Models

Language Models are Few-Shot Learners

Collider bias undermines our understanding of COVID-19 disease risk and severity

Shortcut learning in deep neural networks

Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization

Toward Gender-Inclusive Coreference Resolution

BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

RoBERTa: A Robustly Optimized BERT Pretraining Approach

Invariant Risk Minimization

Causality

PAWS: Paraphrase Adversaries from Word Scrambling

Reducing Gender Bias in Abusive Language Detection

Recognition in Terra Incognita

Gender Bias in Coreference Resolution

Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods

Elements of Causal Inference: Foundations and Learning Algorithms

Invited Commentary: Selection Bias Without Colliders.

Collider scope: when selection bias can substantially influence observed associations

Causal inference and the data-fusion problem

Recovering Causal Effects from Selection Bias

To Adjust or Not to Adjust? Sensitivity Analysis of M-Bias and Butterfly-Bias

Recovering from Selection Bias in Causal and Statistical Inference

Controlling Selection Bias in Causal Inference

Illustrating bias due to conditioning on a collider.

Model index for researchers -openai api

World Economic Forum Global Gender Gap Report

Investigating Gender Bias in Language Models Using Causal Mediation Analysis

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Mind the GAP: A balanced corpus of gendered ambiguous pronouns

Sample selection bias as a specification error

2022. Diversify and

Tsnlp | Test Suites for Natural Language Processing

Github repo to replicate all the plots in this

OpenAI

More General Setting

addresses gender biases and stereotypes, including the assumption of binary gender categories in

Method 2 open source Hugging Face

We parameterize the causal DAGs in Figure 2(a) and (b), with a simple structural causal model (SCM)

9817, Abu Dhabi, United Arab Emirates, December 2022

A Reproducibility Please see the following sources to reproduce methods and measurements in this paper: 1) Method 1 open source Hugging Face Space

Field of Study

Venue Information

Name

Type

URL

Alternate Names