Home Research Papers Datasets State of the Art Pricing

Discover, visualize, and connect AI research papers. Explore the latest trends and insights in artificial intelligence research.

Product

Home
Research Papers
About

Support

Contact
Terms of Service
Privacy Policy

natural-language-processing-3

Probing Language Models

3260 papers • 126 benchmarks • 313 datasets

This task has no description! Would you like to contribute one?

(Image credit: Papersgraph)

Benchmarks

These leaderboards are used to track progress in probing-language-models-3

Trend

Dataset

Best Model

Actions

KAMEL

Libraries

Use these libraries to find probing-language-models-3 models and implementations

Datasets

BioLAMA

KAMEL

Subtasks

No subtasks available.

Most implemented papers

Universal and Independent: Multilingual Probing Framework for Exhaustive Model Interpretation and Evaluation

Oleg Serikov, E. Voloshina, Vitaly Protasov, V. Knyazkova, Tatiana Shavrina•Sun Oct 23 2022

This paper presents and applies the GUI-assisted framework allowing to easily probe massive amounts of languages for all the morphosyntactic features present in the Universal Dependencies data, and proposes a toolkit to systematize the multilingual flaws in multilingual models.

Content

Introduction Benchmarks Datasets Subtasks Libraries Papers

Paper Graph

Quantifying gender bias towards politicians in cross-lingual language models

Isabelle Augenstein, Tiago Pimentel, Karolina Stańczak, Sagnik Ray Choudhury, Ryan Cotterell•Wed Apr 14 2021

The usage of adjectives and verbs generated by language models surrounding the names of politicians as a function of their gender is quantified to demonstrate that pre-trained language models’ stance towards politicians varies strongly across analyzed languages.

27 0

Paper Graph

Probing Toxic Content in Large Pre-Trained Language Models

Dit-Yan Yeung, Nedjma Djouhra Ousidhoum, Yangqiu Song, Tianqing Fang, Xinran Zhao•Thu Dec 31 2020

A method based on logistic regression classifiers is proposed to probe English, French, and Arabic PTLMs and quantify the potentially harmful content that they convey with respect to a set of templates to assess and mitigate the toxicity transmitted by PTL Ms.

146 0

Paper Graph

Probing Language Models for Understanding of Temporal Expressions

Shivin Thukral, Kunal Kukreja, Christian Kavouras•Sat Oct 02 2021

It is found that although large language models fine-tuned on MNLI have some basic perception of the order between points in time, at large, these models do not have a thorough understanding of the relation between temporal expressions.

25 0

Paper Graph

Discontinuous Constituency and BERT: A Case Study of Dutch

Konstantinos Kogkalidis, G. Wijnholds•Tue Mar 01 2022

In this paper, we set out to quantify the syntactic capacity of BERT in the evaluation regime of non-context free patterns, as occurring in Dutch. We devise a test suite based on a mildly context-sensitive formalism, from which we derive grammars that capture the linguistic phenomena of control verb nesting and verb raising. The grammars, paired with a small lexicon, provide us with a large collection of naturalistic utterances, annotated with verb-subject pairings, that serve as the evaluation test bed for an attention-based span selection probe. Our results, backed by extensive analysis, suggest that the models investigated fail in the implicit acquisition of the dependencies examined.

7 0

Paper Graph

The neural architecture of language: Integrative modeling converges on predictive processing

J. Tenenbaum, I. Blank, Evelina Fedorenko, Martin Schrimpf, N. Kanwisher, Greta Tuckute, Carina Kauf, Eghbal A. Hosseini•Fri Jun 26 2020

It is found that the most powerful “transformer” models predict nearly 100% of explainable variance in neural responses to sentences and generalize across different datasets and imaging modalities (functional MRI and electrocorticography).

566 0

Paper Graph

Probing Representations for Document-level Event Extraction

Claire Cardie, Xinya Du, Barry Wang•Sun Oct 22 2023

This work is the first to apply the probing paradigm to representations learned for document-level information extraction (IE), and designed eight embedding probes to analyze surface, semantic, and event-understanding capabilities relevant to document- level event extraction.

2 0

Paper Graph

Adding a benchmark result helps the community track progress.

Probing Language Models | State-of-the-Art