LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models (2023-04-04T00:00:00.000000Z)

TL;DR

LLM-Adapters is presented, an easy-to-use framework that integrates various adapters into LLMs and can execute these adapter-based PEFT methods of LLMs for different tasks, demonstrating that using adapter- based PEFT in smaller-scale LLMs with few extra trainable parameters yields comparable, and in some cases superior, performance to powerful LLMs in zero-shot inference on both reasoning tasks.

Abstract

The success of large language models (LLMs), like GPT-4 and ChatGPT, has led to the development of numerous cost-effective and accessible alternatives that are created by finetuning open-access LLMs with task-specific data (e.g., ChatDoctor) or instruction data (e.g., Alpaca). Among the various fine-tuning methods, adapter-based parameter-efficient fine-tuning (PEFT) is undoubtedly one of the most attractive topics, as it only requires fine-tuning a few external parameters instead of the entire LLMs while achieving comparable or even better performance. To enable further research on PEFT methods of LLMs, this paper presents LLM-Adapters, an easy-to-use framework that integrates various adapters into LLMs and can execute these adapter-based PEFT methods of LLMs for different tasks. The framework includes state-of-the-art open-access LLMs such as LLaMA, BLOOM, and GPT-J, as well as widely used adapters such as Series adapters, Parallel adapter, Prompt-based learning and Reparametrization-based methods. Moreover, we conduct extensive empirical studies on the impact of adapter types, placement locations, and hyper-parameters to the best design for each adapter-based methods. We evaluate the effectiveness of the adapters on fourteen datasets from two different reasoning tasks, Arithmetic Reasoning and Commonsense Reasoning. The results demonstrate that using adapter-based PEFT in smaller-scale LLMs (7B) with few extra trainable parameters yields comparable, and in some cases superior, performance to powerful LLMs (175B) in zero-shot inference on both reasoning tasks.

Authors

References57 items

LMFlow: An Extensible Toolkit for Finetuning and Inference of Large Foundation Models

LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention

GPT-4 Technical Report

LLaMA: Open and Efficient Foundation Language Models

Is ChatGPT a General-Purpose Natural Language Processing Task Solver?

Parameter-Efficient Fine-Tuning Design Spaces

Self-Instruct: Aligning Language Models with Self-Generated Instructions

KronA: Parameter Efficient Tuning with Kronecker Adapter

On the Effectiveness of Parameter-Efficient Fine-Tuning

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model

SparseAdapter: An Easy Approach for Improving the Parameter-Efficiency of Adapters

LST: Ladder Side-Tuning for Parameter and Memory Efficient Transfer Learning

Large Language Models are Zero-Shot Reasoners

OPT: Open Pre-trained Transformer Language Models

PaLM: Scaling Language Modeling with Pathways

Training Verifiers to Solve Math Word Problems

SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer

Exploring Universal Intrinsic Task Subspace via Prompt Tuning

UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuning

Towards a Unified View of Parameter-Efficient Transfer Learning

Finetuned Language Models Are Zero-Shot Learners

LoRA: Low-Rank Adaptation of Large Language Models

Compacter: Efficient Low-Rank Hypercomplex Adapter Layers

The Power of Scale for Parameter-Efficient Prompt Tuning

Are NLP Models really able to Solve Simple Math Word Problems?

Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning

AdapterHub: A Framework for Adapting Transformers

Language Models are Few-Shot Learners

MAD-X: An Adapter-based Framework for Multi-task Cross-lingual Transfer

PIQA: Reasoning about Physical Commonsense in Natural Language

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions

Parameter-Efficient Transfer Learning for NLP

Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge

Attention is All you Need

Program Induction by Rationale Generation: Learning to Solve and Explain Algebraic Word Problems

Solving General Arithmetic Word Problems

MAWPS: A Math Word Problem Repository

Parsing Algebraic Word Problems into Equations

Learning to Solve Arithmetic Word Problems with Verb Categorization

ChatDoctor: A Medical Chat Model Fine-tuned on LLaMA Model using Medical Domain Knowledge

HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face

Crosslingual Generalization through Multitask Finetuning

Recalpaca: Low-rank llama instruct-tuning for recommendation

Stanford alpaca: An instruction-following llama model

Introducing chatgpt

AdaMix: Mixture-of-Adapter for Parameter-efficient Tuning of Large Language Models

Prefix-Tuning: Optimizing Continuous Prompts for Generation

Learn-to-Share: A Hardware-friendly Transfer Learning Framework Exploiting Computation and Parameter Sharing

GPT-J-6B: A 6Billion Parameter Autoregressive Language Model

Intrinsic dimensionality explains the 5262

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

An Adversarial Winograd Schema Challenge at Scale

Socialiqa: Commonsense reasoning about social interactions

math reasoning and commonsense reasoning

2022. Peft: State-of-the-art parameter-eﬃcient ﬁne-tuning methods

2023. Plan-and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models