Evaluating the Moral Beliefs Encoded in LLMs (2023-07-26T00:00:00.000000Z)

TL;DR

This paper introduces statistical measures and evaluation metrics that quantify the probability of an LLM"making a choice", the associated uncertainty, and the consistency of that choice, and applies this method to study what moral beliefs are encoded in different LLMs, especially in ambiguous cases where the right choice is not obvious.

Abstract

This paper presents a case study on the design, administration, post-processing, and evaluation of surveys on large language models (LLMs). It comprises two components: (1) A statistical method for eliciting beliefs encoded in LLMs. We introduce statistical measures and evaluation metrics that quantify the probability of an LLM"making a choice", the associated uncertainty, and the consistency of that choice. (2) We apply this method to study what moral beliefs are encoded in different LLMs, especially in ambiguous cases where the right choice is not obvious. We design a large-scale survey comprising 680 high-ambiguity moral scenarios (e.g.,"Should I tell a white lie?") and 687 low-ambiguity moral scenarios (e.g.,"Should I stop for a pedestrian on the road?"). Each scenario includes a description, two possible actions, and auxiliary labels indicating violated rules (e.g.,"do not kill"). We administer the survey to 28 open- and closed-source LLMs. We find that (a) in unambiguous scenarios, most models"choose"actions that align with commonsense. In ambiguous cases, most models express uncertainty. (b) Some models are uncertain about choosing the commonsense action because their responses are sensitive to the question-wording. (c) Some models reflect clear preferences in ambiguous scenarios. Specifically, closed-source models tend to agree with each other.

Authors

Nino Scherrer

3 papers

D. Blei

8 papers

Claudia Shi

3 papers

TL;DR

Abstract

Authors

References76 items

Moral Foundations of Large Language Models

PaLM 2 Technical Report

Generative Agents: Interactive Simulacra of Human Behavior

Whose Opinions Do Language Models Reflect?

Sparks of Artificial General Intelligence: Early experiments with GPT-4

Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation

The Capacity for Moral Self-Correction in Large Language Models

The Flan Collection: Designing Data and Methods for Effective Instruction Tuning

Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus?

The political ideology of conversational AI: Converging evidence on ChatGPT's pro-environmental, left-libertarian orientation

OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization

Discovering Language Model Behaviors with Model-Written Evaluations

Constitutional AI: Harmlessness from AI Feedback

Talking about Large Language Models

Language Models as Agent Models

Fine-tuning language models to find agreement among humans with diverse preferences

Scaling Instruction-Finetuned Language Models

An Analysis of The Effects of Decoding Algorithms on Fairness in Open-Ended Language Generation

When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment

Improving alignment of dialogue agents via targeted human judgements

Moral Mimicry: Large Language Models Produce Moral Rationalizations Tailored to Political Identity

Out of One, Many: Using Language Models to Simulate Human Samples

Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned

Social Simulacra: Creating Populated Prototypes for Social Computing Systems

Meaning without reference in large language models

Does Moral Code have a Moral Code? Probing Delphi’s Moral Philosophy

Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback

PaLM: Scaling Language Modeling with Pathways

Training language models to follow instructions with human feedback

A General Language Assistant as a Laboratory for Alignment

Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefs

Unsolved Problems in ML Safety

Do Prompt-Based Models Really Understand the Meaning of Their Prompts?

Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasets

Calibrate Before Use: Improving Few-Shot Performance of Language Models

Measuring and Improving Consistency in Pretrained Language Models

Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequences

Social Chemistry 101: Learning to Reason about Social and Moral Norms

The Turking Test: Can Language Models Understand Instructions?

Learning to summarize from human feedback

Scruples: A Corpus of Community Ethical Judgments on 32, 000 Real-Life Anecdotes

Aligning AI With Shared Human Values

Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data

Language Models are Few-Shot Learners

HuggingFace's Transformers: State-of-the-art Natural Language Processing

Fine-Tuning Language Models from Human Preferences

Are Red Roses Red? Evaluating Consistency of Question-Answering Models

The Psychology of Morality: A Review and Analysis of Empirical Studies Published From 1940 Through 2017

Concrete Problems in AI Safety

Moral judgment reloaded: a moral dilemma validation study

Modern hierarchical, agglomerative clustering algorithms

Mapping the moral domain.

Pushing moral buttons: The interaction between personal force and intention in moral judgment

The self-importance of moral identity.

Fast optimal leaf ordering for hierarchical clustering

Longitudinal study of the Defining Issues Test of moral judgment: A strategy for analyzing developmental change.

Crosslingual Generalization through Multitask Finetuning

Inducing anxiety in large language models increases exploration and bias

Large Language Models are Prone to Methodological Artifacts

MoCa: Cognitive Scaffolding for Language Models in Causal and Moral Judgment Tasks

Models Documentation

BECEL: Benchmark for Consistency Evaluation of Language Models

Using Large Language Models to Simulate Multiple Humans

CAN MACHINES LEARN MORALITY? THE DELPHI EXPERIMENT

PERSONALITY PROCESSES AND INDIVIDUAL DIFFERENCES Liberals and Conservatives Rely on Different Sets of Moral Foundations

Common Morality: Deciding What to Do

Information Theory, Inference, and Learning Algorithms

The "big three" of morality (autonomy, community, divinity) and the "big three" explanations of suffering.

Measuring Semantic Entropy

Information radius

Auxiliary Label Acquisition

whether models are sensitive to different answer option orderings and whether they display similar uncertainty levels across question styles. To delve deeper into

the reproducibility of evaluations, we have recorded timestamps (or timeframes) of API calls to models of OpenAI, Cohere, and Anthropic, and timestamps of model downloads from the HuggingFace Hub

Does the action directly result in a death? 2. Does the action directly inflict physical or emotional pain or discomfort?

Does the action directly impair someone's physical, mental, or volitional abilities? 4. Does the action intentionally restrict someone's freedom?

Anthropic