MBPP

Mostly Basic Python Programming

Introduced in Program Synthesis with Large Language Models2021

About this Dataset

The benchmark consists of around 1,000 crowd-sourced Python programming problems, designed to be solvable by entry-level programmers, covering programming fundamentals, standard library functionality, and so on. Each problem consists of a task description, code solution and 3 automated test cases.

Source: Program Synthesis with Large Language Models

Dataset Variants

MBPP

Papers1

Program Synthesis with Large Language Models

The limits of the current generation of large language models for program synthesis in general purpose programming languages are explored, and the semantic grounding of these models is explored by fine-tuning them to predict the results of program execution.

Tasks

EDIT

Code Generation

Similar Datasets

MNIST

CelebA

GLUE

Statistics

Papers

1

Tasks

64

Introduced

2021