miscellaneous-3

Moral Scenarios

3260 papers • 126 benchmarks • 313 datasets

This task has no description! Would you like to contribute one?

(Image credit: Papersgraph)

Benchmarks

These leaderboards are used to track progress in moral-scenarios-3

Trend

Dataset

Best Model

Actions

BIG-bench

Libraries

i

Use these libraries to find moral-scenarios-3 models and implementations

Datasets

BIG-bench

Subtasks

No subtasks available.

Most implemented papers

Scaling Language Models: Methods, Analysis & Insights from Training Gopher

A. Mensch, Katie Millican, Roman Ring, Eliza Rutherford, Jacob Menick, Sebastian Borgeaud, Aida Nematzadeh, O. Vinyals, K. Simonyan, Yujia Li, Igor Babuschkin, Po-Sen Huang, Johannes Welbl, K. Kavukcuoglu, D. Hassabis, James Bradbury, Chris Dyer, L. Sifre, George van den Driessche, Erich Elsen, D. Budden, G. Irving, Laura Weidinger, Maribeth Rauh, Iason Gabriel, William S. Isaac, Saffron Huang, Lisa Anne Hendricks, I. Higgins, Diego de Las Casas, Jack W. Rae, Siddhant M. Jayakumar, Xiang Lorraine Li, Francis Song, Cyprien de Masson d'Autume, Amy Wu, Angeliki Lazaridou, A. Kuncoro, Michela Paganini, Trevor Cai, Jordan Hoffmann, John Aslanides, Sarah Henderson, Susannah Young, Tom Hennigan, Albin Cassirer, Richard Powell, Amelia Glaese, Sumanth Dathathri, Jonathan Uesato, John F. J. Mellor, Antonia Creswell, Nat McAleese, Elena Buchatskaya, Esme Sutherland, Lena Martens, E. Gribovskaya, Domenic Donato, Jean-Baptiste Lespiau, M. Tsimpoukelli, N. Grigorev, Doug Fritz, Thibault Sottiaux, Mantas Pajarskas, Tobias Pohlen, Z. Gong, Daniel Toyama, Tayfun Terzi, Vladimir Mikulik, Aidan Clark, Aurelia Guy, Chris Jones, Matthew G. Johnson, Blake A. Hechtman, Edward Lockhart, Simon Osindero, Laura Rimell, Kareem W. Ayoub, J. Stanway, L. Bennett

Content

•

Tue Dec 07 2021

This paper presents an analysis of Transformer-based language model performance across a wide range of model scales -- from models with tens of millions of parameters up to a 280 billion parameter model called Gopher.

1527 0

Paper Graph

"Oops, Did I Just Say That?" Testing and Repairing Unethical Suggestions of Large Language Models with Suggest-Critique-Reflect Process

Pingchuan Ma, Zongjie Li, Ao Sun, Shuai Wang•Wed May 03 2023

This paper proposes ETHICSSUITE, a test suite that presents complex, contextualized, and realistic moral scenarios to test LLMs, and proposes a novel suggest-critic-reflect (SCR) process, serving as an automated test oracle to detect unethical suggestions.

15 0

Paper Graph

Evaluating the Moral Beliefs Encoded in LLMs

Nino Scherrer, D. Blei, Claudia Shi, Amir Feder•Tue Jul 25 2023

This paper introduces statistical measures and evaluation metrics that quantify the probability of an LLM"making a choice", the associated uncertainty, and the consistency of that choice, and applies this method to study what moral beliefs are encoded in different LLMs, especially in ambiguous cases where the right choice is not obvious.

211 0

Paper Graph

MOKA: Moral Knowledge Augmentation for Moral Event Extraction

Lu Wang, Xinliang Frederick Zhang, Winston Wu, Nick Beauchamp•Wed Nov 15 2023

This work annotates a new dataset, **MORAL EVENTS**, and proposes **MOKA**, a moral event extraction framework with **MO**ral **K**nowledge **A**ugmentation, which leverages knowledge derived from moral words and moral scenarios to produce structural representations of morality-bearing events.

12 0

Paper Graph

SaGE: Evaluating Moral Consistency in Large Language Models

P. Kumaraguru, Vamshi Bonagiri, S. Vennam, Priyanshul Govil, Manas Gaur•Tue Feb 20 2024

An information-theoretic measure called Semantic Graph Entropy (SaGE), grounded in the concept of “Rules of Thumb” (RoTs) to measure a model’s moral consistency is proposed, which reveals that task accuracy and consistency are independent problems, and there is a dire need to investigate these issues further.

0 0

Paper Graph

Adding a benchmark result helps the community track progress.