PG-19

Introduced in Compressive Transformers for Long-Range Sequence Modelling

About this Dataset

A new open-vocabulary language modelling benchmark derived from books.

Source: Compressive Transformers for Long-Range Sequence Modelling

Dataset Variants

PG-19

Papers1

Compressive Transformers for Long-Range Sequence Modelling

The Compressive Transformer is presented, an attentive sequence model which compresses past memories for long-range sequence learning and can model high-frequency speech effectively and can be used as a memory mechanism for RL, demonstrated on an object matching task.

Dataset Loaders

EDIT

🔥

huggingface/datasets

tfpytorchjax

🔥

huggingface/datasets

tfpytorchjax

🔥

huggingface/datasets

tfpytorchjax

🎯