LLaMA: Open and Efficient Foundation Language Models
Published in
arXiv.org(2023)External Links:
TL
TL;DR
LLaMA, a collection of foundation language models ranging from 7B to 65B parameters, is introduced and it is shown that it is possible to train state-of-the-art models using publicly available datasets exclusively, without resorting to proprietary and inaccessible datasets.
Abstract
We introduce LLaMA, a collection of foundation language models ranging from 7B to 65B parameters. We train our models on trillions of tokens, and show that it is possible to train state-of-the-art models using publicly available datasets exclusively, without resorting to proprietary and inaccessible datasets. In particular, LLaMA-13B outperforms GPT-3 (175B) on most benchmarks, and LLaMA-65B is competitive with the best models, Chinchilla-70B and PaLM-540B. We release all our models to the research community.
References80 items
1
OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization
2
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
3
Solving Quantitative Reasoning Problems with Language Models
4
Emergent Abilities of Large Language Models
5