GPT Understands, Too (2021-03-18T00:00:00.000000Z)

TL;DR

A novel method P-Tuning is proposed that employs trainable continuous prompt embeddings in concatenation with discrete prompts that stabilizes training by minimizing the gap between various discrete prompts, and improves performance by a sizeable margin on a wide range of NLU tasks including LAMA and SuperGLUE.

Authors

Jie Tang

8 papers

Xiao Liu

9 papers

Zhilin Yang

9 papers

Zhengxiao Du

5 papers

Yanan Zheng

3 papers

Yujie Qian

3 papers

Ming Ding

3 papers

TL;DR

Authors

References48 items

FewNLU: Benchmarking State-of-the-Art Methods for Few-Shot Natural Language Understanding

Pre-Trained Models: Past, Present and Future

Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity

The Power of Scale for Parameter-Efficient Prompt Tuning

Learning How to Ask: Querying LMs with Mixtures of Soft Prompts

Factual Probing Is [MASK]: Learning vs. Learning to Recall

Zero-Shot Text-to-Image Generation

Calibrate Before Use: Improving Few-Shot Performance of Language Models

Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm

What Makes Good In-Context Examples for GPT-3?

Making Pre-trained Language Models Better Few-shot Learners

Language Models are Open Knowledge Graphs

X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Models

Interventional Few-Shot Learning

It’s Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners

Self-Supervised Learning: Generative or Contrastive

Revisiting Few-sample BERT Fine-tuning

Language Models are Few-Shot Learners

5分で分かる!? 有名論文ナナメ読み：Jacob Devlin et al. : BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding

How Context Affects Language Models' Factual Predictions

How Can We Know What Language Models Know?

BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Language Models as Knowledge Bases?

Commonsense Knowledge Mining from Pretrained Models

RoBERTa: A Robustly Optimized BERT Pretraining Approach

The CommitmentBank: Investigating projection in naturally occurring discourse

XLNet: Generalized Autoregressive Pretraining for Language Understanding

A Multiscale Visualization of Attention in the Transformer Model

What Does BERT Look at? An Analysis of BERT’s Attention

A Structural Probe for Finding Syntax in Word Representations

Unified Language Model Pre-training for Natural Language Understanding and Generation

SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems

BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions

Transformer-XL: Attentive Language Models beyond a Fixed-Length Context

ReCoRD: Bridging the Gap between Human and Machine Commonsense Reading Comprehension

A Convergence Theory for Deep Learning via Over-Parameterization

WiC: 10, 000 Example Pairs for Evaluating Context-Sensitive Representations

Looking Beyond the Surface: A Challenge Set for Reading Comprehension over Multiple Sentences

GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding

Choice of Plausible Alternatives: An Evaluation of Commonsense Causal Reasoning

The Winograd Schema Challenge

Crafting Papers on Machine Learning

Prefix-Tuning: Optimizing Continuous Prompts for Generation

Autoprompt: Eliciting knowledge from language models with automatically generated prompts

Language Models are Unsupervised Multitask Learners

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

The PASCAL Recognising Textual Entailment Challenge

Field of Study

Journal Information

Name

Page

Volume

Venue Information

Name

Type

URL